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Abstract 

A  literature  review  focusing  on  the  identification  of  useful  engineering 
design  guidelines  for  adaptive  aiding  systems  was  conducted.  Approximately 
40  articles  were  reviewed,  and  over  140  design  guidelines  were  extracted. 
Sources  for  the  guidelines  included  concept  development,  empirical 
investigations,  or  analysis  of  fielded  systems,  and  literature  reviews  in  adaptive 
aiding  technology.  A  two-dimensional  taxonomic  structure  was  developed  and 
used  to  categorize  the  guidelines.  The  first  dimension  of  the  taxonomy  is  based 
on  Rouse’s  (1988)  framework  for  adaptive  aid  design.  The  second  dimension 
focuses  on  the  design  implication  of  each  of  the  identified  guidelines.  This 
structure  allows  for  efficient  categorization  of  the  information  and  assists  in 
showing  where  research  results  exist  to  help  the  designer  of  adaptive  aiding 
systems.  The  guidelines  are  presented  with  rationale  for  each  guideline 
included.  The  guidelines  and  supporting  material  in  each  cell  of  the  taxonomy 
are  analyzed  for  usefulness.  Recommended  areas  of  future  research  and 
analysis  of  the  current  research  status  in  adaptive  aiding  is  discussed. 
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1.0  Introduction 

The  complexity  of  modern  engineering  systems  (e.g.,  tactical  aircraft, 
spacecraft,  and  process  control  systems)  and  the  tasks  that  operators  must 
perform  within  these  systems  have  led  to  a  boom  of  automation  technologies  to 
assist  the  operator  in  dealing  with  these  emerging  complexities.  The 
shortcoming  of  most  of  these  technologies  is  that  although  they  allow  higher 
performance  on  specific  tasks,  they  introduce  systems  complexity  problems  of 
their  own.  Often,  instead  of  reducing  operator  workload  and  increasing  system 
performance  through  the  use  of  automation  technology,  the  automation  only 
serves  to  increasB  workload  on  the  operator  and  dBcreass  situation  awareness. 

Originally,  function  aiding  (e.g.,  Fitts,  1951)  approaches  were  used  by 
system  designers  as  the  automation  philosophy.  However,  researchers  and 
designers  alike  have  since  realized  that  the  demands  of  highly  automated 
modern  systems  (e.g.,  tactical  aircraft)  on  the  operator  vary  greatly  overtime 
and  task  loading.  Often  these  systems  demand  an  inordinate  amount  of 
operator  attention.  What  is  needed  is  a  technology  that  integrates  the 
automated  functions  and  considers  the  operator's  demands  in  a  dynamic 
fashion.  In  other  words,  the  automation  must  be  flexible  to  handle  manifold 
performance  demands  and  be  responsive  to  both  the  changing  needs  of  the 
system  as  well  as  the  human  operator. 

One  concept  introduced  to  mitigate  the  effects  of  increasing  automation 
in  systems  is  adaptive  aiding  (also  referred  to  as  "adaptive  automation"  and 
"adaptive  function  allocation").  Adaptive  aiding  is  a  systems  automation 
philosophy  that  proposes  the  use  of  automation  to  assist  the  operator  when 
system  performance  is  likely  to  degrade  past  the  point  of  acceptability  at  some 
point  in  the  near  future  (Rouse  and  Rouse,  1983).  The  viability  of  adaptive 
aiding  for  system  control  has  been  discussed  (Rouse,  1988)  and  demonstrated 
(Andes.  1987;  Lind,  1989)  in  recent  endeavors. 

An  increasing  number  of  automated  systems  are  being  introduced  Into 
the  aerospace  and  process  control  domains.  To  provide  the  optimal  dynamic 
function  aiding  and  human-aid  interaction  characteristics,  a  wide  breadth  of 
applicable  human  performance  and  systems  design  literature  must  be 
considered  by  the  designer.  A  large  base  of  research  results,  conceptual 
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development,  and  lessons  learned  contribute  to  the  necessary  knowledge  for 
state  of  the  art  aid  design. 

Research  in  adaptive  aiding  has  been  accumulating  over  the  past  two 
decades.  Much  of  this  work  includes  conceptual  development  and  empirical 
Investigation  in  limited  experimental  domains.  However,  a  substantial  amount 
of  this  research  is  the  post  hoc  evaluation  of  implemented  systems  or  design 
methods.  Inherent  to  this  work  are  guidelines  and  suggestions  that  would  be 
applicable  to  the  design  of  future  aiding  systems.  These  guidelines  are  not 
often  readily  accessible  to  the  designer  because  they  are  embedded  within 
technical  reports. 

Rouse  (1988)  has  compiled  a  number  of  relevant  principles  for  adaptive 
aiding  design  in  a  review  of  the  current  state  of  the  art  in  adaptive  aiding 
technology.  That  collection,  however,  represents  only  a  small  sampling  of  the 
potential  guidelines  that  would  be  useful  for  designers  of  adaptive  aiding 
systems. 

The  current  literature  review  was  performed  for  several  reasons.  The  primary 
effort  was  to  assess  the  sophistication  of  design  guidelines  for  adaptive  aiding 
systems.  Other  reasons  were  to  determine  the  usefulness  of  these  guidelines  and  to 
assess  the  level  of  empirical  support  for  the  application  of  these  guidelines  to  system 
design.  The  ultimate  goal  was  to  determine  the  directions  for  research  that  would  be 
most  appropriate  given  the  current  state  of  adaptive  aiding  system  design.  To  this 
end,  this  document  includes  a  review  of  those  sources  believed  to  be  the  most 
representative  reference  materials  used  in  the  design  of  aiding  systems.  This 
document  is  an  important  step  in  the  production  of  a  comprehensive  set  of 
engineering  guidelines  for  designers. 

This  document  is  organized  into  six  sections.  After  the  introduction, 

Section  2  defines  the  scope  and  application  of  the  guidelines  presented 
throughout  the  document.  Sources  from  several  scientific  domains  (i.e.,  human 
performance  literature,  automated  systems  design,  human-computer  interaction 
studies,  etc.)  were  consulted  for  the  production  of  design  guidelines.  Core 
design  guidelines  were  extracted  from  the  references  included  in  this  paper, 
justified  where  appropriate,  and  categorized  in  a  design-oriented  taxonomy. 

Section  3  explains  the  taxonomy  developed  for  guideline  organization. 

The  taxonomy  was  produced  to  highlight  the  type  of  research  that  has  been 
conducted,  to  organize  the  available  design  knowledge,  and  to  identify  where 
more  work  needs  to  be  done. 
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Section  4  presents  the  guidelines  arranged  in  the  taxonomy  according  to 
supporting  rationale.  Each  subsection  presents  a  group  of  guidelines  and 
discusses  them  In  relation  to  the  corresponding  design  issue.  A  rationale  for 
each  guideline  is  supplied  as  necessary,  and  implications  for  system  design  are 
addressed. 

The  summary  and  discussion  in  Section  5  analyzes  the  guideline  groups 
in  terms  of  sufficiency  for  design.  Additionally,  significant  but  lacking  design 
information  Is  identified,  and  suggestions  for  obtaining  requisite  design 
knowledge  are  proposed. 

Section  6,  the  final  section  of  this  review,  discusses  the  current  state  of 
the  art  In  aiding  design.  This  section  addresses  what  types  of  research  should 
be  directed  to  provide  the  knowledge  that  will  empower  the  designer  to  produce 
optimal  adaptive  aiding  systems. 


2.0  Scope  and  application  of  this  document 

Literature  relevant  to  the  design,  implementation,  and  evaluation  of 
adaptive  aiding  systems  spans  several  areas  of  computer  science,  systems 
engineering,  and  human  factors  psychology.  The  current  review  is  an  initial 
step  in  the  production  of  a  comprehensive  set  of  engineering  guidelines  for  aid 
design.  As  such,  approximately  40  articles  covering  the  aforementioned  areas 
have  been  reviewed.  One  hundred  forty-two  design  guidelines  have  been 
extracted:  these  are  enumerated  and  discussed  in  this  document. 

The  articles  reviewed  in  preparation  of  this  document  represent  the  prime 
reference  material  used  by  adaptive  aiding  designers  at  Search  Technology. 
This  material  is  viewed  similarly  in  the  government  and  aerospace  design 
communities.  System  implementations  and  evaluations,  and  interaction  and 
communication  studies  were  reviewed  to  establish  a  range  of  guidelines  that 
address  the  numerous  issues  faced  by  designers  of  aiding  systems.  The 
guidelines  extracted  from  these  sources  discuss  conceptual  approaches  to 
aiding  in  terms  of  system  design  and  user-aid  interaction  characteristics.  The 
review  was  not  exhaustive,  but  it  covers  several  areas  of  concern. 

There  are  two  primary  approaches  to  operator  aiding:  adaptive  aiding 
(Rouse  and  Rouse,  1983),  and  function  aiding  (Fitts,  1951;  Lind,  1989; 

Krobusek  et  al.  1985).  Both  approaches  are  considered  in  this  review,  since  the 
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designer  must  decide  which  approach  he  will  take  early  In  the  systems 
requirements  process.  Note  that  we  cannot  determine  the  correct  philosophy: 
The  design  context  may  dictate  what  level  of  automation  flexibility  is 
appropriate.  Further,  system  requirements  often  determine  whether  the  system 
will  be  task-centered  (i.e.,  function  aiding)  or  human-requirements  centered 
(i.e.,  adaptive  aiding)  (see  Andes  and  Rouse,  1991  for  an  in-depth  discussion  of 
this  issue). 

It  is  important  to  mention  that  the  current  review  is  not  merely  a  repeat  of 
Rouse's  1988  Human  Factors  article  "Adaptive  aiding  for  human/computer 
control."  The  current  paper  attempts  to  integrate  Rouse’s  summarized  material 
with  Information  sources  of  a  more  applied  genre.  However,  for  organizational 
purposes,  this  review  does  rely  on  Rouse's  Framework  for  Design  (originally 
proposed  in  the  1 988  review  article). 

The  framework  for  design  proposes  six  fundamental  aiding  design 
questions  for  the  designer  and  suggests  several  possible  answers  to  those 
questions.  This  review  uses  Rouse's  framework  as  one  dimension  of  an 
organizational  taxonomy  for  design  guidelines  (this  approach  is  quite  rational 
when  viewed  from  the  designer's  perspective).  For  example,  each  of  the 
questions  must  be  addressed  to  ensure  that  the  design  is  the  best  possible  for 
the  current  application.  Further,  by  supplying  guidelines  for  each  of  the 
questions,  we  facilitate  the  design  process  by  allowing  the  designer  to  use  the 
experience  of  others,  even  if  the  current  application  does  not  closely  resemble 
the  reference  context.  The  second  dimension  of  the  taxonomy  considers 
implications  of  the  guidelines  on  the  design  process.  The  framework  and 
taxonomy  are  reviewed  in  Section  3. 

We  have  attempted  to  provide  as  much  specific  information  as  possible 
in  each  of  the  guidelines.  It  is  a  relatively  simple  matter  to  produce  a  high  level 
guideline  that  provides  little  utility  to  an  applied  designer.  The  more  difficult  task 
is  to  extract  useful  design  information  from  the  literature,  represent  it  in  terms 
that  applications  engineers  can  understand,  and  indicate  the  domain  of 
applicability  for  the  guidelines  that  result.  We  hope  that  this  document  provides 
a  useful  start  on  that  process. 

Although  this  review  is  written  from  the  psychological  perspective  of 
human-aid  interaction,  it  addresses  the  engineering  design  community 
foremost.  The  reader  should  note  that  most  of  the  guidelines  have  1  supporting 
empirical  investigation  at  best,  and  some  of  that  supporting  material  may  be  of 
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a  speculative  nature.  In  addition,  the  generalizability  of  some  of  the  guidelines 
is  questionable,  but  worth  mention.  Nonetheless,  the  applicable  material  and 
supporting  literature  content  of  the  guideline  are  covered  as  completely  as 
possible.  Relevant  material  will  be  presented  to  support  this  viewpoint. 


3.0  Adaptive  Aiding  Design  Guideiine  Taxonomy 

Engineering  design  guidelines  must  address  specific  issues  to  be  useful 
to  the  designer.  As  stated  earlier,  the  present  review  has  produced  142 
individual  adaptive  aiding  design  guidelines  from  various  sources.  A  serial 
listing  of  these  guidelines  does  not  provide  much  utility.  In  fact,  such  an 
arrangement  may  actually  reduce  utility  and  confuse  a  designer  who  is 
attempting  to  apply  relevant  information  to  the  design  problem  at  hand. 

One  major  requirement  for  any  classification  scheme  chosen  is  that  it 
support  the  engineering  design  perspective.  Another  requirement  is  that  it 
cover  all  issues  faced  during  the  engineering  design  process.  Further,  this 
classification  scheme  should  support  expansion  as  research  in  the  field  of 
adaptive  aiding  progresses. 

Several  approaches  to  guideline  classification  were  identified  and 
evaluated  for  the  current  review.  The  initial  guideline  organization  process 
involved  partitioning  the  guidelines  into  two  types  of  design  principles: 
Principles  of  adaptation  and  principles  of  interaction.  Principles  of  adaptation 
prescribe  when  and  how  adaptation  occurs.  Alternatively,  principles  of 
interaction  relate  to  operator  acceptance  of  the  aiding.  This  distinction  was 
initially  applied  to  the  list  of  guidelines,  however,  it  became  apparent  that  the 
resulting  principles-based  categorization  was  insufficient  in  characterizing  the 
robustness  of  the  guidelines. 

Since  the  aiding  system  design  community  is  the  intended  audience  of 
the  guidelines  in  this  document,  the  Framework  for  Design  proposed  by  Rouse 
(1 988 )  was  chosen  as  the  taxonomy  for  organizing  the  guidelines.  Within  this 
framework,  a  structured  set  of  conceptual  design  issues  are  systematically 
addressed.  Though  context  specificity  is  not  directly  addressed,  the  framework 
provides  the  aid  designer  with  an  outline  of  the  range  of  possible  design 
alternatives  and  information  that  designers  may  use  In  choosing  among  them. 
The  questions  posed  within  Rouse's  framework  for  design  are  discussed  below. 
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whflt  is  adaptfid  to?  -  The  designer  of  the  aiding  system  must  determine  which 
system  entities  are  adapted  and  which  are  left  in  a  fixed  state.  Either  the  operator, 
the  system,  or  the  aid  may  adapt  in  a  given  situation.  There  are  also  several  levels 
within  each  entity  that  may  be  adapted.  For  example,  a  class  of  operator,  a  particular 
operator,  or  a  particular  operator  in  a  specific  situation  may  be  adapted  to 
accommodate  environmental  circumstances  and  the  present  task.  Furthermore,  the 
aid  may  adapt  to  the  user  and/or  the  task. 

Who  does  tha  adantino?  -  The  designer,  the  aid,  or  the  system  operators  may 
invoke  adaptation  in  a  given  situation.  The  aid  should  initiate  adaptation  if  the 
system's  definition  of  adaptation  must  be  refined,  or  if  operators  are  unlikely  to 
perceive  the  need  for  adaptation.  Additionally,  the  entity  that  invokes  adaptation 
must  interact  with  either  static  or  dynamic  adaptation  within  the  system.  Static 
methods  of  adaptation  are  determined  by  design  specifications  before 
implementation,  and  do  not  change  during  system  use.  Dynamic  adaptation  is 
based  on  changeable  environmental,  system,  or  operator  conditions,  and  is  typically 

associated  with  the  image  of  adaptive  aiding. 

Whfin  does  adaptatinn  occur?  -  Adaptation  may  occur  off-line  (prior  to  system 
operation)  or  on-line  (according  to  changing  system  demands).  Additionally,  an  aid 
may  determine  that  a  pilot  needs  assistance,  but  will  wait  until  a  proper  time  to 
perform  the  aiding.  Adaptive  aiding  should  be  invoked  whenever  pilot  workload  and 
performance  demands  change. 

What  methods  of  adaptation  aoDlv?  -  There  are  three  primary  methods  of 
adaptation  within  adaptive  aiding.  Transformation  involves  changing  a  task  to  make 
performance  of  that  task  easier  for  the  operator.  Partitioning  allows  the  operator  to 
share  performance  of  particluar  tasks  with  the  aiding  system.  Allocation  Is  the  the 
most  common  and  the  easiest  type  of  aid  to  implement,  where  the  aid  either  performs 
a  task  itself  or  allocates  that  task  to  the  operator. 

How  i.Q  adaptation  done?  -  Adaptation  may  be  approached  by  measurement 
of  operator  or  system  performance,  or  by  modeling  operator  resources,  intentions,  or 
system  performance.  These  methods  determine  aiding  needs  within  a  system  by 
addressing  how  inputs  affect  making  the  decision  to  aid. 

What !.«;  the  nature  of  aid-operator  communication?  -  This  addresses  the 
information  exchanged  between  operator  and  aid  about  the  effects  of  adaptation. 
Explicit  communication  provides  ample  Information  exchange,  but  has  high  interface 
and  Interaction  costs.  These  costs  include  demands  on  the  pilot's  time  and  cognitive 
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and  attentional  resources.  Implicit  communication  does  not  Impose  a  high  cost,  but 
may  be  ambiguous  and  result  in  communication  error. 

A  thorough  and  accurate  classification,  however,  required  the  inclusion  of  a 
second  taxonomic  dimension:  implications  of  the  guidelines  on  system 
implementation.  This  second  dimension  enabled  us  to  classify  guidelines  in  terms  of 
their  influence  on  system  performance,  workload,  user  acceptance  of  the  aid,  and 
situation  awareness.  We  have  integrated  the  two  dimensions  (design  Issues  and  the 
effects  of  guidelines  on  system  behavior)  in  a  6  x  4  matrix  to  obtain  a  24-cell 
taxonomy  for  guideline  categorization. 

The  resulting  taxonomic  arrangement  for  the  guidelines  contained  within  this 
review  is  depicted  in  Table  1 .  The  six  design  questions  compose  the  rows  of  the 
taxonomy,  and  the  four  ways  in  which  guidelines  may  affect  adaptive  aiding  systems 
compose  the  columns.  In  Section  4,  Adaptive  Aiding  Design  Guidelines,  we  present 
the  guidelines  within  the  taxonomy  and  discuss  the  possible  implications  of  the 
guidelines  on  system  design  and  behavior.  We  have  assigned  a  unique  tag  to  each 
guideline  to  help  the  reader  understand  how  the  guidelines  fit  into  the  taxonomy. 

This  tag  has  four  parts:  the  section  number  (4;  Section  4  contains  all  guidelines),  a 
row  number  (1  through  6,  corresponding  to  the  six  design  questions),  a  column 
number  (1  through  4,  corresponding  to  the  four  effects  of  implemented  guidelines  on 
the  aiding  system),  and  a  lower  case  letter  unique  to  each  guideline  in  a  given  cell  of 
the  taxonomy.  For  example,  guideline  4.3.2.a  is  discussed  in  terms  of  the  "When 
does  adaptation  occur?"  design  issue  (row  3  of  the  taxonomy)  and  the  implication  of 
this  guideline  on  workload  (column  2  of  the  taxonomy).  In  other  words,  this  guideline 
addresses  how  the  timing  of  adaptation  affects  workload  within  a  system.  Finally,  the 
"a"  signifies  that  this  guideline  Is  the  first  discussed  in  this  particular  cell  of  the 
taxonomy. 
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DESIGN  FRAMEWORK 

USER 

SITUATION 

QUESTION 

PERFORMANCE 

WORKLOAD 

ACCEPTANCE 

ASSESSMENT 

What  is  adapted  to? 

4.1.1 

4.1.2 

4.1.3 

4.1 '4 

Who  does  the  adapting? 

4.2.1 

4.2.2 

4.2.3 

4.2.4 

When  does  the  adaptation 
occur? 

4.3.1 

4.3.2 

4.3.3 

4.3.4 

What  methods  of  adaptation 
apply? 

4.4.1 

4.4.2 

4.4.3 

4.4.4 

How  is  adaptation  done? 

4.5.1 

4.5.2 

4.5.3 

4.5.4 

What  is  the  nature  of  operator- 
aid  communication? 

4.6.1 

4.6.2 

4.6.3 

4.6.4 

Table  1  -  Design  Guideline  Classification  Taxonomy 


4.0  Adaptive  aiding  design  guidelines 

The  guidelines  extracted  during  the  review  process  are  presented  in  this 
section,  categorized  according  to  the  taxonomy  introduced  in  Section  3.  Ttiere  are 
six  major  section  headings,  each  of  which  corresponds  to  one  of  the  six  framework 
for  design  questions.  These  six  sections  are  further  divided  into  four  subsections  that 
address  the  design  implications  for  the  guidelines.  The  subsections  discuss  the 
effects  of  guidelines  on  the  resulting  design  in  terms  of  performance,  workload, 
operator  acceptance,  and  the  ability  to  maintain  situational  awareness.  The 
guidelines,  rationale  for  the  guidelines,  and  relationships  between  guidelines  are 
then  discussed  with  supporting  information  included  where  needed. 


4.1  Guidelines  for  "What  is  Adapted  To?" 

This  section  addresses  the  object  of  adaptation.  In  an  adaptive  aiding 
scenario,  the  task  may  be  adapted  to  make  it  easier  for  the  human  to  perform. 
Alternatively  the  human  may  be  adapted  through  training  to  perform  the  task  more 
proficiently.  The  effects  of  beneficial  adaptation  will  be  manifested  as  improvements 
in  system  performance,  achievement  of  optimal  levels  of  operator  workload, 
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increases  in  operator  acceptability  of  an  aiding  system,  and  increases  in  the  ability  of 
an  operator  to  achieve  situation  awareness. 

4.1.1  Implications  on  Performanca 

The  operator  or  the  aiding  system  may  be  adapted  in  any  aiding  situation. 
Therefore,  the  performance  of  a  system  is  largely  affected  by  the  actions  of  the 
operator  or  the  aid  in  response  to  adaptation.  This  Implies  that  design  decisions 
require  the  consideration  of  the  effects  of  adaptation  on  system  performance.  This 
section  presents  guideiines  that  address  how  adaptation  of  an  agent  affects  system 
performance.  In  particular,  it  discusses  the  effects  of  the  choice  of  interface  format, 
development  of  aiding  system  knowledge  structures,  adaptation  to  user  capabilities, 
allocation  of  tasks,  and  the  process  of  aiding  on  system  performance  output.  The 
guidelines  are  enumerated  below. 

Guidelines 

4.1,1. a  The  selected  knowledge  representation  scheme  for  aid  should  be  able  to 

provide  a  framework  for  representing  system  task  knowledge.  One 
approach  is  using  scripts,  plans,  and  actions  (Shank  and  Abelson,  1977). 
Another  is  to  use  state  transition-action  diagrams  to  affect  system  state 
through  aiding  (Andes,  1987). 

Explanation:  The  manner  in  which  operator  knowledge  and  ability  are 
represented  during  design  of  an  aid  may  affect  how  well  the  Implemented 
aid  ultimately  performs.  During  design  it  is  Important  to  consider  the  tasks 
that  are  to  be  aided  and  context  under  which  aiding  will  occur.  According 
to  Andes  (1987),  the  knowledge  representation  system  within  the  aiding 
system  should  account  for  knowledge  that  operators  use  during  task 
execution.  The  choice  of  knowledge  representation  scheme  depends  on 
aiding  context,  the  number  of  decisions  required  in  a  procedure,  and  the 
level  of  control  achievable  by  the  aid  in  the  system. 

Empirical  Support: 
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4.1 .1  .b  Use  adaptive  interfaces  to  support:  metaphoric  consistency,  short-term 
memory  support,  maintenance  of  user  context,  context  customization, 
learning  acceleration,  and  error  recognition  (Norcio  and  Stanley,  1989). 

Explanation:  This  guideline  addresses  the  conceptual  stages  of  adaptive 
interface  design.  For  example,  when  a  pilot  is  learning  to  incorporate 
performance  of  a  given  task  with  other  tasks,  the  information  in  the 
interface  should  be  explicit.  As  the  pilot  becomes  proficient  in 
incorporating  all  tasks,  the  interface  should  be  less  explicit  to  avoid 
providing  the  pilot  with  Information  that  he  no  longer  needs  to  perform 
those  tasks.  This  guideline  is  in  accord  with  good  design  of  display 
interfaces  for  computer  software  in  general,  and  would  be  well-suited  to 
the  design  of  adaptive  aiding  systems.  An  illustrative  example  of 
guidelines  4.1.1. a  and  4.1. 1.b  can  be  found  in  Rouse,  Geddes,  and  Curry 
(1987). 

Empirical  Support: 

4.1.1.C  Designers  are  interested  in  producing  consistent  behavior  of  aiding 
behavior  in  normal  vs.  novel  situations  (Andes  and  Rouse,  1991). 

Explanation:  Andes  and  Rouse  have  found  that  designers  of  aiding 
systems  are  interested  In  system  reliability  in  both  normal  and  novel 
situations.  Although  one  goal  of  designers  is  to  build  systems  that 
successfully  aid  operators  in  highly  likely  situations,  designing  systems 
with  expandable  functionality  is  an  additional  goal  of  significant 
importance.  Thus,  designers  are  interested  in  knowing  how  an  aid  will 
perform  in  situations  outside  of  the  intended  functional  envelope. 

Empirical  Support:  The  study  by  Andes  and  Rouse  (1991)  addressed  what 
kind  of  information  aiding  designers  value  in  the  development  of  aiding 
specifications.  Statistical  analyses  of  designers’  preferences  showed  that 
designers  were  interested  in  reliability  of  the  aid  on  tasks  other  than  those 
for  which  the  aid  was  originally  designed. 
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4.1. 1. d  Use  adaptive  interfaces  to  support:  mixed  dialogue  initiative,  vigilance 

support,  navigational  support,  progressive  disclosure,  and  regulation  of 
control  and  display  surfaces  (Norcio  and  Stanley,  1989). 

Explanation:  Note  that  these  features  are  related  to  changes  in  the 
appearance  of  the  system  to  the  user  over  time.  By  including  these 
features  within  an  aiding  system,  the  designer  insures  that  the  system 
adapts  to  user's  needs.  Although  this  and  the  next  guideline  were 
originally  designed  for  computer  interfaces,  they  address  some  of  the 
same  issues  present  in  the  design  in  display  interfaces  for  adaptive  aiding 
systems. 

Empirical  Support: 

4.1. 1. e  The  designer  of  decision  support  systems  must  derive  the  following  from 

decision  makers  in  the  beginning  stages:  list  of  user  objectives  Bnd 
objectives  hierarchy,  list  of  alternative  decisions,  and  a  list  of  outcomes  for 
each  alternative  (Sage  and  White,  1984). 

Explanation:  Knowledge  representation  should  consider  the  operator 
information  requirements  and  purpose  of  the  aid  as  well.  Designers  of 
decision  support'systems  must  derive  information  from  operators  in  the 
initial  stages  of  design.  This  information  should  include  a  list  of  user 
objectives  and  an  objectives  hierarchy,  a  list  of  alternative  decisions,  and  a 
list  of  possible  outcomes  for  each  alternative.  Such  an  analysis  of  the 
requirements  of  decision  support  systems  enables  the  designer  to 
evaluate  alternative  plans  and  decisions  efficiently.  Further,  it  allows  the 
designer  to  discover  and  utilize  existing  dominance  patterns  among 
alternatives. 

Empirical  Support:  Weisbrod,  Davis  and  Freedy  (1977)  provide  an 
empirical  analysis  of  dynamic  decision  processes.  They  have  determined 
that  direct  judgement  techniques,  measurement  of  user  utility  values,  and 
dynamic  utility  estimation  are  useful  ways  to  estimate  and  update  the 
decision  maker’s  utility  functions  for  decisions.  Their  work  is  discussed  in 
more  detail  in  Section  4.1.3. 
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4.1 .1  .f  Use  flexible  automation  as  a  means  for  increasing  the  proficiency  of  novice 
users  and  for  preventing  frustration  that  may  occur  with  overly  complex 
systems  (Norcio  and  Stanley,  1989). 

Explanation:  Adaptive  aiding  is  useful  in  increasing  performance  of 
operators  with  different  skills.  Variable  adaptation  should  be  used  in 
adaptive  aiding  systems  to  increase  the  proficiency  of  the  novice  user,  and 
to  prevent  frustration  that  may  otherwise  occur  with  the  use  of  complex 
systems. 

Empirical  Support:  Although  this  design  guideline  is  commonly  held  as 
true,  it  is  in  need  of  validation. 

4.1. 1. g  The  long-term  impact  of  automation  on  pilot  skills  should  be  considered 

early  in  the  design  process.  Effective  training  programs  must  also  be 
designed  to  maximize  the  pilot’s  contribution  to  mission  effectiveness 
(Parasuraman,  Bahri,  Deaton,  Morrison  and  Barnes,  1990). 

Explanation:  This  implies  that  in  highly  automated  systems,  pilot  skills  may 
degrade  if  they  are  not  practiced.  The  long-term  impact  of  automation 
should  be  considered  early  in  design  since  it  is  easier  to  include 
embedded  training  in  the  system  during  the  design  process.  Further,  the 
purpose  of  effective  system-use  training  programs  should  be  to  maximize 
the  pilot’s  contribution  to  mission  effectiveness. 

Empirical  Support:  This  guideline  is  widely  accepted  In  the  operational 
community  and  is  probably  not  necessary  to  validate. 

4.1. 1. h  Training  programs  in  adaptive  aiding  should  stress  operator-aid  interaction 

skills  and  cognitive/problem  solving  skills  rather  than  psychomotor  skills 
(Parasuraman  et  al.  1990). 

Explanation:  Inadequate  training  may  lead  to  widespread  automation- 
induced  pilot  skill  decrements.  Furthermore,  automation  necessitates  a 
shift  from  psychomotor  skills  to  cognitive  and  problem  solving  skills. 
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Training  must  augment  cognitive  and  problem  solving  skills  so  that  pilots 
maintain  the  ability  to  handle  complex  problem  situations  when  automation 
fails  or  when  problem  dimensions  are  out  of  the  aid's  range. 

Empirical  Support: 

4.1 .1  .i  Allocation  of  tasks  to  man  or  machine  depends  on  the  state  of  the  system 
and  the  state  of  the  world  (Boys,  1990).  Within  each  system  function, 
different  levels  of  system  autonomy  (LOA)  are  possible. 

Explanation:  Task  allocation  (the  "what"  that  is  adapted  to)  has  been 
analyzed  from  a  variety  of  directions.  Boys  claims  that  optimal  allocation  of 
tasks  to  man  or  machine  depends  on  the  state  of  the  system  and  the  state 
of  the  world.  The  way  tasks  are  prioritized  is  affected  by  contextual  factors 
which  change  over  time.  Another  factor  that  affects  task  allocation  Is  the 
level  of  aiding  autonomy  assigned  to  a  system  by  the  designer  or  the 
operator.  A  system  with  total  autonomy  will  assume  performance  of  all 
tasks.  Alternatively,  a  system  with  no  autonomy  assumes  the  operator  will 
perform  all  tasks  unless  the  operator  off-loads  some  of  them  to  the  system. 
The  levels  of  autonomy  (LOA)  concept  has  been  developed  by  Krobusek, 
Boys  and  Palko  (1989),  and  is  discussed  in  more  detail  in  Section  4.4.1. 

Empirical  Support:  See  Krobusek,  Boys  and  Palko  (1989)  for  further 
support  of  this  task  allocation  guideline. 

4,1. 1.j  Beware  of  basing  allocation  decisions  solely  on  computer  aid  abilities. 
Human's  aptitudes,  cognitive  styles,  and  attitudes  may  affect  behavior 
across  situations  (Morris,  Rouse,  Ward  and  Frey,  1984). 

Explanation:  Humans'  abilities  should  be  considered  and  used  during 
allocation  decisions  because  those  abilities  are  a  valuable  resource  to 
system  control.  It  must  be  remembered  though,  that  humans'  aptitudes, 
cognitive  styles,  and  attitudes  may  affect  their  behavior  across  situations, 
and  that  behavior  as  such  may  not  be  totally  reliable.  The  Fitts’  type  list  of 
human  abilities  that  would  result  from  basing  allocation  decisions  on 
computer  abilities  would  not  account  for  individual  differences  between 
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humans,  however.  Such  a  list  would  therefore  not  optimize  task  allocation 
decisions. 

Empirical  Support: 

4.1. 1.k  A  balance  between  task  difficulty  and  automation  must  be  found  so  that  the 
human  is  performing  at  an  optimal  level  of  task  stimulation;  the  task  should 
be  neither  too  difficult  (overarousing)  nor  too  easy  (underarousing) 
(Morrison,  Gluckman  and  Deaton,  1990). 

Explanation:  Humans  perform  best  over  a  limited  range  of  task  stimulation. 
If  the  effort  required  is  too  high,  the  human  may  fail  to  do  the  task.  If  the 
effort  required  is  too  low,  the  human  may  suffer  vigilance  decrements. 

Empirical  Support:  This  conclusion  relates  to  the  work  of  Sen  (1984), 
where  the  balance  of  information  input  to -a  human  decision  maker  was 
analyzed  empirically.  Sen  found  that  optimal  human  performance 
required  a  certain  level  of  task  difficulty,  and  automation  of  the  remaining 
tasks. 

4.1 .1 .1  Beware  of  creating  a  passive,  complacent  operator  through  use  of 

automation  for  routine  tasks  if  the  operator  is  responsible  for  intervention 
during  system  failure.  Consider  involving  the  operator  in  these  tasks  as  an 
active  agent  and  using  embedded  training  (Parasuraman  et  al.  1990). 

Explanation:  Since  a  passive  operator’s  skills  are  susceptible  to 
complacency  and  boredom,  the  operator  may  not  be  capable  of  taking- 
over  tasks  if  the  system  fails.  Parasuraman  et  al.  suggest  that  designing 
embedded  training  into  the  system  will  keep  the  operator  actively  involved 
In  system  activities. 

Empirical  Support:  Several  studies  have  found  that  performance  of  boring 
tasks  decreases  with  Increased  performance  time.  This  guideline  is  well- 
supported  by  human  performance  literature. 
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4.1 .1  .m  Users  may  adapt  to  the  aid  and/or  the  task  depending  on  their  functional 
models  of  the  aid  and  their  own  level  of  expertise  (Rouse,  1988). 

Explanation:  Designers  must  be  able  to  determine  how  operators' 
conceptual  models  of  aiding  systems  and  their  levels  of  expertise  affect 
adaptation  because  humans  adapt  to  everything. 

Empirical  Support: 

4.1 .1  .n  In  multiple  task  situations,  it  is  more  desirable  to  aid  a  few  tasks  with 
emphasis  on  increased  performance  than  to  aid  the  prioritization  of  all 
tasks  to  be  completed  by  the  operator  (Derrick,  1988). 

Explanation:  Derrick  found  that  human  and  system  performance  was 
better  when  a  few  tasks  were  aided  than  when  the  human  performed  tasks 
according  to  schedule  and  task  priorities. 

% 

Empirical  Support:  This  was  supported  by  the  findings  of  Wickens  and 
Yeh  (1983),  where  subjective  effort  ratings  focused  more  on  processing 
requirements  than  performance  outcomes.  Basically,  humans  are  more 
concerned  with  what  they  have  to  do  rather  than  how  well  they  do  it.  In 
other  words,  humans  are  good  at  prioritization  --  task  performance  support 
will  yield  better  results  than  support  of  task  scheduling. 

4.1 .1 .0  Attempt  to  spread  operator  resource  demands  over  the  largest  number  of 
resource  structures  in  dual-task  environments.  This  is  consistent  with 
Multiple  Resource  Theory  and  will  yield  smaller  performance  decrements 
than  concentrating  processing  demands  on  a  minimal  number  of 
resources  (Derrick,  1988). 

Explanation;  In  accord  with  Multiple  Resource  Theory  (MRT),  Derrick 
suggests  that  in  dual-task  environments,  operator  resource  demands  are 
spread  over  as  many  human  processing  resource  structures  as  possible. 
This  procedure  minimizes  human  performance  decrements  because  it 
prevents  the  overload  of  one  particular  processing  resource.  The  visual 
system  or  the  auditory  system  are  examples  of  processing  resources. 
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Empirical  Support:  In  experiments,  Derrick  found  the  greatest  performance 
decrements  in  dual-task  pairs  where  processing  demands  were  spread 
over  a  minimal  number  of  resource  structures.  In  order  to  minimize 
performance  decrements  then,  demands  on  an  operator's  resources 
should  be  distributed  among  the  largest  number  of  resources. 

4.1 .1  .p  System  response  time  to  operator  action  must  not  degrade  system 

effectiveness.  Changes  in  response  tirne  should  be  considered  following 
changes  in  task  allocation  responsibility,  and  to  identify  the  time  required 
to  reconfigure  the  system  (Krobusek,  Boys  and  Palko,  1989). 

Explanation:  Krobusek,  Boys  and  Palko  (1989)  discuss  the  general 
requirements  for  successful  aiding  system  performance.  Of  the  most 
important  requirements  is  minimizing  system  response  time.  This  may  be 
difficult  to  achieve,  however,  because  changes  in  task  allocation  may 
increase  system  response  time  in  two  ways.  First,  the  pilot  must  spend 
some  amount  of  time  reconfiguring  task  allocations,  and  second,  any 
reallocation  may  affect  the  ability  of  the  system  to  respond  promptly. 
Designers  should  consider  the  effects  of  system  response  time  on  system 
effectiveness  during  the  design  process.  In  addition,  Andes  (1990)  and 
Morrison,  Gluckman,  and  Deaton  (1990)  have  discussed  the  criticality  of 
maintaining  response  time  with  regard  to  hardware  and  intervention 
methods. 

Empirical  Support: 

4.1 .1  .q  Systems  that  relax  control  requirements  improve  both  control  and 
subsystem  performance,  but  systems  (e.g.,  on  required  control 
performance)  that  relax  subsystem  requirements  improve  only  subsystem 
performance.  Consider  this  result  when  constructing  the  top-level  design 
for  the  system  (Chu  and  Rouse,  1979). 

Explanation:  One  factor  that  should  be  considered  early  in  the  design 
process  is  the  system  capacity  to  relax  control  requirements.  However, 
systems  that  relax  subsystem  requirements  improve  only  subsystem 
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performance;  control  performance  is  not  improved.  This  was  attributed  to 
the  fact  that  control  tasks  have  priority  over  subsystem  tasks,  and  that 
control  task  inefficiency  is  likely  to  affect  subsystem  task  performance. 

Empirical  support:  The  experiment  performed  by  Chu  and  Rouse  provides 
support  for  this  guideline. 

4.1.2  Implications  on  Workload 

i 

Pilot  workload  is  affected  by  changes  in  the  status  of  current  tasks,  and  by  the 
introduction  or  removal  of  other  tasks.  If  the  pilot  is  the  entity  that  adapts  in  a  given 
situation,  he  may  experience  a  temporary  or  long-term  increase  in  workload.  The 
pilot  must  learn  to  manipulate  the  system  to  his  benefit  and  to  Incorporate  new 
external  information  into  the  performance  of  current  tasks.  If  alternatively,  the  aid 
adapts,  the  pilot  is  free  to  proceed  with  current  tasks,  and  may  not  even  be  aware  of 
the  aid's  activity.  In  situations  where  the  aid  adapts  to  either  the  pilot's  actions  or  the 
task,  pilot  workload  is  unlikely  to  Increase.  Guidelines  having  implications  for  pilot 
workload  are  enumerated  below. 

Guidelines 

4.1. 2.a  Pilot  workload  should  be  optimized  rather  than  simply  minimized  in 

adaptive  automation  systems  (Parasuraman  et  al.  1990). 

Explanation:  Apparently,  operators  are  unable  to  remain  focused  on  tasks 
that  involve  extremely  low  levels  of  workload,  and  are  consequently 
unable  to  perform  these  tasks  well.  An  optimal  level  of  workload  should  be 
assigned  to  the  operator  to  insure  his  active  participation  in  system 
functioning.  Methods  of  workload  optimization  will  be  specifically 
addressed  in  section  4.5.2. 

Empirical  Support:  This  guideline  has  been  reflected  in  the  results  of 
several  studies  conducted  in  the  fields  of  vigilance  and  automation 
performance  deficit.  It  has  been  shown  that  both  extremely  high  and  low 
workload  situations  degrade  task  performance. 
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4.1 .2. b  At  low  rates  of  information  input  to  the  operator,  mean  decision  time  (time 

required  to  decide  what  output  corresponds  with  the  input)  is  a  linear 
function  of  information  input  (Sen,  1984). 

Explanation:  The  work  of  Sen  has  provided  insight  into  the  differences 
between  the  human  decision  maker  and  the  mathematically  ideal  decision 
maker.  His  work  also  addresses  how  these  differences  influence  the 
distribution  of  workload  in  decision  support  environments.  This  guideline 
states  that  for  low  information  input  rates,  the  higher  the  input  rate,  the 
longer  it  takes  to  make. a  response  decision.  Furthermore,  the  relationship 
between  input  rate  and  decision  time  is  linear. 

Empirical  Support:  Sen's  work  provides  direct  support  for  this  guideline. 

4.1. 2. C  At  high  rates  of  information  input  to  the  operator,  information  overload 

occurs,  and  the  linear  function  of  information  input  breaks  down  (Sen, 
1984). 

Explanation:  The  typical  human  decision  maker  experiences  information 
overload  at  high  information  input  rates.  This  is  evidenced  by  the 
breakdown  of  the  linear  function  between  information  Input  rate  and 
information  processing  time;  the  human  is  no  longer  able  to  keep  decision¬ 
making  pace  with  the  rate  of  input. 

Empirical  Support:  The  functions  produced  by  Sen  are  useful  in  modeling 
information  inputs  for  human-aid  interaction. 

4.1 .2. d  Input  to  a  decision  maker  can  be  partitioned  into  groups  depending  on  Its 

characteristics.  When  input  approaches  overload,  the  decision  maker 
Ignores  certain  characteristics  and  prioritizes  the  input  with  regard  to  the 
most  important  characteristics  (Sen,  1 984). 

Explanation:  This  behavior  helps  reduce  the  possibility  of  overload 
because  It  minimizes  the  amount  of  input  that  the  decision  maker  must 
account  for. 
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Empirical  Support:  Research  by  Miller  (1969)  supports  this  guideline  by 
showing  that  the  number  of  decision  maker  errors  increases  significantly 
during  input  overload  situations.  The  decision  maker  is  unable  to  consider 
all  the  Information  required  for  making  correct  decisions. 

4.1 .2.6  Random  omission  of  information  pertinent  to  the  decision  process  occurs 
when  the  decision  maker  simply  does  not  make  a  decision  when  he  is 
given  certain  conditions.  This  occurs  with  low  frequency  at  low  and 
medium  information  input  rates,  but  with  significantly  high  frequency  at 
high  input  rates  (Sen,  1984). 

Explanation:  Since  the  decision  maker  cannot  account  for  all  information 
provided  to  him  during  increasingly  high  input  rates,  he  may  omit  certain 
pieces  of  information  from  consideration  on  a  random  basis.  Errors  that 
result  from  this  type  of  information  processing  strategy  are  called  random 
omission  errors.  These  errors  occur  infrequently  at  low  and  medium 
information  input  rates.  However,  they  occur  with  much  greater  frequency 
at  high  information  input  rates. 

Empirical  Support:  Miller’s  (1969)  work  provides  support  for  this  guideline. 

4.1. 2.f  In  incomplete  decision  responses  under  high  workload,  the  decision 

maker  specifies  input  within  a  set  of  bounds  and  takes  a  partial  course  of 
action.  This  is  a  conscious  overload  avoidance  strategy  in  which  the 
decision  maker  uses  only  part  of  relevant  input  available;  the  decision 
maker  exhibits  error  behavior  in  order  to  avoid  overload  (Sen,  1984). 

Explanation:  Incomplete  decision  responses  occur  when  decision  makers 
consider  only  the  information  that  has  particular  characteristics,  or  falls 
within  a  certain  range  of  topics.  The  decision  maker  makes  response 
decisions  after  considering  only  part  of  the  input,  and  therefore  makes  only 
partial  responses.  Like  random  error  behavior,  incomplete  decision 
response  is  a  conscious  overload  avoidance  strategy  that  occurs  at  • 
medium  and  high  information  input  rates. 
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Empirical  Support:  A  clear  explanation  for  this  phenomenon  does  not 
exist. 

4.1 .2. g  Try  to  preserve  "cognitive  unity"  when  transforming  a  task.  Cognitive  unity 

refers  to  the  operator's  perception  that  the  transformed  task  is  the  same  as 
the  original  task  (Andes,  1990). 

Explanation:  This  guideline  applies  directly  to  the  design  of  adaptive 
aiding  systems.  When  an  aided  task  supports  cognitive  unity,  the  operator 
perceives  the  transformed  task  to  be  functionally  the  same  as  the  original 
task.  Andes  promotes  the  preservation  of  cognitive  unity  to  make  aided 
tasks  easier  to  perform. 

Empirical  Support:  Although  transformation  aiding  implies  that  the  two 
tasks  are  the  same,  the  human  operator  may  actually  perceive  two  different 
tasks.  This  is  expected  to  lead  to  an  increase  in  workload,  but  such  an 
effect  has  not  been  empirically  evaluated. 

4.1. 2. h  Pilots  can  be  trained  to  reconfigure  task  allocation  dynamically  using 

levels  of  automation  (LOA)  approach.  Task  allocation  should  be  based  on 
sensory,  cognitive,  and  behavioral  capabilities  of  the  pilot  population 
(Krobusek  et  al.  1 989). 

Explanation:  Regardless  of  automation  approach,  researchers  in  the 
adaptive  aiding  field  commonly  espouse  this  guideline.  Task  allocation 
should  be  based  on  sensory,  cognitive,  and  behavioral  capabilities  of  the 
pilot  population  in  order  to  minimize  the  negative  effects  of  workload  on 
operator  performance.  In  addition,  pilots  should  be  trained  to  account  for 
these  capabilities  since  they  may  be  responsible  for  changing  task 
allocation  configurations  during  adaptation. 

Empirical  Support: 

4.1. 2.1  In  a  multiple  alternative  decision  support  environment,  use  decision 
analysis  to  support  ranking  of  alternative  decisions  based  on  the 
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dominance  structure  of  the  decision  maker’s  prioritized  alternatives  (Sage 
and  White,  1984). 

Explanation:  Sage  and  White  suggest  that  designers  use  decision 
analysis  to  enable  ranking  of  alternative  decisions  to  support  multiple 
alternative  decision  making.  This  ranking  should  be  based  on  the 
dominance  structure  of  the  decision  maker's  prioritized  alternatives.  The 
purpose  of  this  analysis  is  to  decrease  interaction  time  and  increase 
overall  system  efficiency  for  each  Individual  operator.  In  their  paper,  Sage 
and  White  provide  explicit  algorithms  for  performing  such  analyses. 

Empirical  Support: 

4.1. 2.j  Apply  aiding  during  air-to-air  combat  with  mission-phase  tailoring. 

Mission-phase  tailoring  defines  specific  tasks  (functions)  that  are 
automated  as  a  function  of  the  mission  phase  (i.e.,  intercept,  within  visual 
range  combat  maneuvering,  weapons  delivery,  disengagement)  (e.g., 
Lind,  1989). 

Explanation:  In  air-to-air  combat,  workload  may  be  minimized  by  aiding 
through  mission-phase  tailoring.  This  type  of  aid  tailoring  defines  specific 
tasks  that  are  automated  as  a  function  of  the  mission  phase.  Lind  has 
found  that  introducing  aiding  in  a  mission-phase  tailored  manner  fosters 
user  acceptance  of  the  aid  and  minimizes  workload  because  the  aided 
tasks  are  predefined  and  the  pilot  is  aware  of  his  task  responsibilities  at  all 
times. 

Empirical  Support: 

4.1.3  Implications  on  Operator  Acceptance 

Operator  acceptance  of  the  aid  is  critical  to  the  successful  Incorporation  of 
adaptive  aiding  systems  in  the  cockpit.  If  user  acceptance  is  considered  in 
decisions  of  what  is  adapted  to  in  the  system,  the  relative  abilities  of  different  users 
must  also  be  considered.  Investigators  have  begun  to  develop  guidelines  in  order  to 
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achieve  user  acceptance  of  aiding,  however  not  all  of  these  guidelines  have  been 
empirically  validated. 


Guidelines 

4.1. 3. a  Designers  are  interested  in  information  about  the  appropriate  motivations 

for  different  types  of  aiding  (Andes  and  Rouse,  1991). 

Explanation:  Andes  and  Rouse  have  determined  that  the  designer's 
approach  to  the  acceptance  issue  is  to  consider  information  about  the 
appropriate  motivations  for  different  types  of  aiding.  Designers  promote 
aid  acceptance  by  deciding  what  type  of  aiding  would  be  desirable  to  the 
operator  in  a  particular  context. 

Empirical  Support:  Andes  and  Rouse  (1991)  provides  support  for  this 
guideline. 

4.1 .3. b  Two  levels  of  operator  preference  information  necessary  for  aid  tailoring 

are:  population  preferences  and  individual  operator  preferences  (Andes, 
1990). 

Explanation:  Operator  acceptance  of  an  aid  depends  on  whether  operator 
preferences  are  accounted  for  during  design.  The  appropriate  mix  of 
population  preferences  and  individual  operator  preferences  that  results  in 
optimal  aid  tailoring  and  user  acceptance  is  not  yet  known.  It  Is  likely  that 
the  aid-operator  interaction  will  provide  Information  relating  the  the 
necessary  level  of  tailoring. 

Empirical  Support: 

4.1 .3. C  Incorporate  models  within  the  aid  that  allow  predictions  of  the  relative 

abilities  of  users  and  the  aid  to  perform  the  task  In  particular  situations 
(Rouse,  1988). 
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Explanation:  Rouse  has  suggested  that  operator  acceptance  would  be 
fostered  if  the  aid  were  capable  of  predicting  whether  the  user  or  the  aid. 
would  better  perform  a  task  in  a  given  situation.  Furthermore,  aiding 
strategies  that  are  based  on  user  populations  are  likely  to  be  accepted 
because  the  relative  abilities  of  users  and  aids  are  accounted  for.  Thus, 
operator  ability  provides  the  basis  for  task  allocation.  The  incorporation  of 
predictive  performance  models  within  the  aid  are  necessary  to  determine 
whether  the  aid  or  the  operator  will  better  perform  a  task. 

Empirical  Support:  Lehner  et  al.  (1987)  and  Moms  and  Rouse  (1986) 
have  also  addressed  the  issue  of  operator  acceptance  in  the  context  of 
task  allocation. 

4.T.3.d  Dynamic  adaptation  of  the  interface  to  the  user  may  be  attained  by  utilizing 
information  provided  to  the  system  through  user  interactions  with  it  in  a 
specific  context  (Norcio  and  Stanley,  1989). 

Explanation:  Norcio  and  Stanley  have  found  that  user  Interactions  provide 
information  that  can  be  used  by  the  system  to  allow  dynamic  adaptation  of 
the  interface  to  the  user.  These  Interactions  should  be  paired  with  the 
contexts  in  which  they  were  performed  if  they  are  to  be  properly  used  as 
the  basis  for  adaptation.  The  dynamic  adaptation  approach  is  similar  to 
using  operator  intent  as  the  basis  for  adaptation,  and  is  believed  to 
promote  operator  acceptance  of  an  aid.  Rouse,  Geddes  and  Curry  (1987) 
provide  an  analogous  approach  to  supporting  user  acceptance  of  aiding  in 
complex  systems  by  determining  user  intent. 

Empirical  Support: 

4.1 .3.e  Estimating  user  utility  values:  Use  direct  judgement  technique  to  get  a 

value  for  each  decision  outcome.  Combine  these  single  utilities  linearly  to 
produce  the  decision  maker's  utilities  for  complex  decisions  (Weisbrod  et 
al.  1977). 

Explanation:  Dynamic  parameter  estimation  is  a  viable  method  of  attaining 
operator  acceptance  of  decision  support  aids.  Weisbrod,  Davis  and 
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Freedy  (1977)  have  evaluated  this  approach  with  an  implemented 
decision  aiding  tool,  ADDAM,  and  have  developed  three  guidelines  which 
may  apply  well  to  adaptive  aiding  systems  (guidelines  4.1.3.e  to  4.1.3.g). 
This  guideline  suggests  using  direct  judgement  to  estimate  user  utilities  for 
specific  decision  outcomes.  The  single  utilities  can  be  combined  linearly 
to  produce  utilities  for  complex  decisions.  Both  single  and  complex  utility 
values  can  be  used  to  determine  the  best  decision  support  system  output 
fora  for  each  user. 

Empirical  Support:  The  Weisbrod  et  al.  study  supports  this  guideline  by 
showing  improved  operator  performance  with  this  type  of  decision  aiding. 
Madni  (1988)  provides  another  example  of  the  dynamic  parameter 
estimation  technique. 

4.1 .3. f  Estimate  user  utility  values  for  decision  output  by  inference  from  behavlQl 

in  a  Rimnifi  namble.  The  decision  maker  responds  to  simple  gambles  with 
monetary  rewards.  The  choices  form  a  database  from  which  utility  values 
are  Inferred  dynamically  (Weisbrod  et  al.  1977). 

Explanation:  User  utility  values  can  be  inferred  by  pooling  a  user's 
behavior  in  several  simple  gamble  situations.  Utilities  may  thus  be  inferred 
dynamically  from  these  user  behaviors.  The  utility  values  can  then  be 
used  to  determine  the  best  decision  support  output  in  differing  decision 
contexts.  This  technique  was  shown  to  be  highly  useful  during  dynamic 
adaptive  aiding  situations. 

Empirical  Support:  see  4.1. 3. e. 

4.1 .3. g  Use  Dynamic  utility  estimation  to  estimate  and  update  the  decision  maker's 

utility  functions  for  decisions  by  observing  alternative  decision  preference 
in  context  and  dynamically  reconstructing  utility  functions  based  on 
changing  decision  maker  behavior  (Weisbrod  et  al.  1977). 

Explanation:  Dynamic  utility  estimation  in  decisiori  support  systems  may 
foster  user  acceptance  because  the  decision  maker's  utility  functions  are 
continually  estimated  and  updated.  The  user  does  not  get  the  impression 
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that  the  system  is  basing  its  aid  in  utility  estimations  that  are  no  longer 
accurate.  Dynamic  estimation  can  be  accomplished  in  aiding  systems  that 
record  alternative  decision  preferences  in  specific  contexts  and  continually 
reconstruct  utility  functions  based  on  changing  decision  maker  behavior. 

Empirical  Support:  see4.1.3.e. 

4.1. 3.h  Decision  making  systems  using  dynamic  utility  estimation  techniques 

foster  user  acceptance  because  operators  feel  that  the  aiding  is  based  on 
their  own  values,  giving  them  a  high  degree  of  apparent  control  (Weisbrod 
etal.  1977). 

Explanation:  When  decision  support  Is  an  important  part  of  the  adaptive 
aid,  the  designer  can  embed  dynamic  utility  estimation  in  the  system  to 
optimize  decision  support  output.  If  the  aid  is  based  on  changing  user 
preference  structures,  the  user  is  more  likely  to  approve  of  the  aid.  This  is 
related  to  aiding  tailorability,  which  is  an  important  issue  in  the  adaptive 
aiding  field. 

Empirical  Support: 

4.1.4  Implications  on  Situation  Assessment 

Whether  the  operator,  the  aid,  or  the  task  is  adapted  in  a  given  situation  affects 
the  ability  of  either  the  operator  or  the  aid  to  assess  the  current  situation.  If  the 
operator  is  adapted,  he  may  be  unable  to  achieve  situation  assessment  because  he 
is  engaged  in  adapting  tasks.  If  the  aid  is  adapted,  the  operator  is  free  to  consider 
information  about  the  current  situation,  however,  the  aid  may  be  unable  to  account 
for  new  information  about  the  current  status  of  the  system.  Further,  if  the  task  is 
adapted,  the  operator  may  be  preoccupied  with  performing  the  new  task  and  may  not 
be  able  to  effectively  integrate  new  information.  It  is  important  to  also  consider 
research  that  has  been  conducted  on  situation  assessment  abilities  of  the  aid  and 
the  operator  In  this  section. 

Guidelines 


25 


Contract  No.  F33615-88-C-3612 
Report  No.  NAWCADWAR-92085-60 

4.1. 4. a  Four  domains  of  knowledge  are  necessary  in  any  operator  aiding  system: 

knowledge  of  the  current  user,  knowledge  of  interaction  scheme, 
knowledge  of  problem  task,  and  underlying  operational  knowledge  (Norcio 
and  Stanley,  1989). 

Explanation:  In  their  literature  review,  Norcio  and  Stanley  concluded  that 
the  above  four  domains  of  knowledge  are  necessary  in  any  operator 
aiding  system.  User  knowledge  includes  users'  cognitive  limitations  and 
strengths,  perceptual  weaknesses  and  strengths,  problem  solving 
strategies,  attentional  allocation,  and  mental  models  of  the  aiding  system. 
Knowledge  of  the  interaction  scheme  can  be  inferred  from  the  interface 
format.  Problem  task  knowledge  can  be  inferred  through  task  modeling 
based  on  the  system’s  performance  of  the  task,  and  goal  detection  and 
plan  inference  based  on  the  user  dialogue  context  and  the  environment. 
Finally,  the  underlying  system  knowledge  can  be  achieved  through 
optimizing  the  input  and  output  within  the  boundaries  of  system's  limits. 
Although  knowledge  requirements  may  not  change  for  the  user  of  a 
particular  interface  In  a  particular  situation,  they  are  likely  to  change  for  the 
aiding  system  throughout  a  pilot's  mission.  Such  changing  conditions 
should  prompt  the  adaptive  aiding  system  to  continuously  and  dynamically 
assess  which  of  the  four  types  of  information  (user,  interaction,  problem 
task  and  system  information)  are  available  for  analysis.  Andes  (1987) 
discusses  these  issues  from  the  implementation  perspective. 

Empirical  Support:  This  guideline  was  extracted  from  a  literature  review; 
the  original  sources  provided  the  validation  for  each  conclusion. 

4.1 .4. b  The  system  must  allow  the  pilot  to  efficiently  adjust  attentional  resources 

within  a  pre-specified  amount  of  time  to  allow  him  to  intervene  if  the  system 
fails.  This  is  primarily  an  operator  information  requirements  issue 
(Morrison,  Gluckman  and  Deaton,  1990). 

Explanation:  Pilot  situation  assessment  abilities  must  be  supported  by  any 
aiding  system.  Further,  the  system  must  allow  the  pilot  enough  time  to 
adjust  his  attention  so  that  he  can  intervene  efficiently  before  the  system 
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fails  completely.  The  operator,  like  the  aid,  requires  certain  information  at 
particular  points  in  time  to  maintain  acceptable  levels  of  system  operation. 

Empirical  Support:  Both  Logan  (1990)  and  Gluckman  (1990)  have  noted 
that  the  time  required  is  variable  for  operator  adjustment  to  workload 
changes  during  system  failure.  This  issue  needs  further  investigation. 

4.1. 4.C  The  cost  of  attending  to  one  subsystem  and  ignoring  the  others  is  a 

function  of  the  changes  in  subsystem  states  that  occur  during  the  time  that 
it  is  ignored  (Greenstein  and  Revesman,  1986). 

Explanation:  An  operator’s  ability  to  assess  situations  is  affected  by  the 
degree  to  which  he  attends  to  some  aiding  subsystems  and  ignores  others. 
Additionally,  situation  assessment  ability  is  influenced  by  the  number  of 
changes  that  a  occur  in  a  subsystem  while  it  is  Ignored.  Since  operators 
cannot  directly  observe  system  states,  they  must  be  estimated  through 
system  displays.  In  order  to  perform  this  estimation,  the  operator  must 
focus  on  one  subsystem  and  is  therefore  unable  to  monitor  all  active 
displays.  The  longer  a  given  subsystem  is  Ignored,  the  more  It  may 
change.  Consequently,  the  cost  incurred  by  ignoring  that  subsystem 
increases  as  well. 

Empirical  Support:  More  researcTi  is  necessary  to  determine  how 
inferences  of  subsystem  states  may  be  optimized  through  decision  aiding. 

4.1:  Section  Summary 

This  section  discussed  the  implications  of  what  is  adapted  in  an  aiding  system 
based  on  different  aspects  of  system  performance.  The  guidelines  emphasize  the 
importance  of  adaptive  interfaces,  the  use  of  appropriate  knowledge  representation 
schemes,  operator  decision  support,  and  the  consideration  of  pilot  expertise  and  the 
aid's  effects  on  the  pilot  in  the  design  of  aiding  systems.  These  guidelines  suggest 
that  designers  should:  a)  implement  aiding  systems  to  optimize  pilot  workload  and 
task  allocation  throughout  each  mission,  and  b)  allow  operators  to  dynamically  tailor 
the  aid  to  personal  preferences  and  abilities. 
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4.2  Guidelines  for  "Who  Does  the  Adapting?" 

Researchers  have  addressed  the  Issue  of  whom  is  the  appropriate  agent  of 
adaptation.  In  several  research  examples,  the  human  operator  adapts  to  the 
situation,  particularly  in  the  case  of  novel  contexts.  The  designer  must  consider  the 
inherent  abilities  of  both  operator  and  aid,  and  make  decisions  about  who  should 
initiate  aiding  accordingly. 

The  operator  or  the  aid  may  use  two  types  of  adaptation  for  any  situation  in  the 
environment.  These  are  static  and  dynamic  adaptation.  Static  adaptation  implies 
that  Information  is  considered  once,  and  then  an  adaptation  decision  is  made. 
Dynamic  adaptation  implies  that  there  is  a  continuing  account  of  system  status,  and 
that  the  adaptation  occurs  with  regard  to  changes  In  this  information.  Generally,  in  a 
given  system,  if  the  process  of  adaptation  must  be  refined  and/or  changed  and  it  is 
unlikely  that  the  operator  will  perceive  the  need  for  change,  then  the  aid  is  the 
appropriate  agent  of  adaptation.  Otherwise  the  operator  should  initiate  adaptation. 

4.2.1  Implications  on  Performance 

In  aiding  systems,  either  the  aid  may  be  adapted  to  the  operator,  or  the 
operator  may  be  adapted  to  the  aid.  Undoubtedly,  system  performance  will  be 
affected  depending  on  which  agent  is  selected  for  adaptation.  The  goal  of  the 
designer  is  to  determine  how  to  which  agent  will  best  perform  a  given  task  based  on 
contextual  and  predictive  information. 

Guidelines 

4.2.1. a  Task  allocation  default  mode  should  be  the  human,  with  the  option  to 

delegate  tasks  to  automation  when  he  is  unable  to  make  allocation 
decisions  within  the  time  available  (Morris  and  Rouse,  1985). 

Explanation:  This  guideline  suggests  that,  in  order  to  optimize  aiding 
system  performance,  the  human  operator  allocate  tasks  unless  he  does 
not  have  enough  time  to  make  effective  decisions.  Three  facts  support  this 
guideline.  First,  current  technology  does  not  permit  automation  of  all 
existing  types  of  system  control.  Second,  coherence  of  the  human's 
overall  role  in  the  system  is  promoted  when  the  human  makes  allocation 
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decisions.  Third,  humans  are  more  likely  to  accept  this  approach  than 
others  because  it  gives  them  a  high  degree  of  apparent  system  control. 

Empirical  Support:  Morris  and  Rouse’s  (1985)  study  provides  support  for 
this  guideline. 

4.2.1.  b  Effective  use  of  an  aid  requires  that  humans  are  able  to  determine  when 

and  how  the  aid  should  be  used,  and  when  it  should  not  be  used  (Morris, 
Rouse,  Ward,  and  Frey,  1985). 

Explanation:  If  the  operator  knows  when  and  how  to  use  an  aid,  it  is 
acceptable  to  allow  him  to  be  in  charge  of  applying  the  aid.  However, 
when  the  operator  does  not  know  when  and  how  to  use  an  aid,  automated 
decision  aiding  should  be  invoked.  This  will  allow  optimization  of  system 
control  and  aiding  application. 

Empirical  Support:  The  Morris  et  al.  study  supports  this  guideline.  The 
operator  control  of  aiding  issue  is  also  discussed  by  Morris,  Rouse  and 
Frey  (1985),  and  Lehneret  al.  (1987). 

4.2.1.  c  Availability  of  aiding  in  a  system  can  affect  performance  positively  even 

when  the  aid  is  not  in  use  (Morris  and  Rouse,  1986). 

Explanation:  Morris  and  Rouse  have  shown  that  operator  performance 
increases  with  the  simple  availability  of  an  aiding  system.  This  is  true 
regardless  of  whether  or  not  the  aiding  system  Is  used.  Thus,  the 
operator's  perception  of  being  in  control  of  the  aiding  system  may  have  a 
critical  effect  on  operator  and  system  performance. 

Empirical  Support:  The  study  provides  support  for  this  guideline. 

4.2.1  .d  The  costs  associated  with  task  automation  may  be  slowed  response  to 
unexpected  events  while  monitoring  the  automation  (Parasuraman  et  al. 
1990). 
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Explanation:  When  operators  are  required  to  change  their  interaction  with 
a  system  from  passive  monitoring  to  active  engagement,  their  responses 
are  slower  than  if  they  had  not  been  required  to  passively  monitor  the 
system.  This  shows  that  the  degree  of  task  automation  can  affect  human 
operator  performance  as  well  as  system  performance.  Furthermore, 
responses  to  unexpected  events  will  be  especially  slowed,  not  only 
because  operators  have  been  monitoring  system  activity,  but  because  they 
are  not  prepared  for  such  events. 

Empirical  Support:  Related  to  this  guideline,  Bortolussi  and  Vidulich  (in 
press)  have  found  that  pilots  prefer  manual  control  over  high  priority  tasks 
like  weapon  selection,  weapon  delivery,  and  flight  control.  During  critical 
events,  slowed  responses  to  these  tasks  could  jeopardize  the  mission. 

4.2.1  .e  Tasks  that  are  least  likely  to  be  voluntarily  performed  by  the  pilot  (i.e., 
complex,  difficult  tasks)  should  be  automated  if  possible  (Morrison  et  al. 
1990). 

Explanation:  In  dynamic  task  allocation  situations,  tasks  that  pilots  are 
least  likely  to  perform  should  automatically  be  allocated  to  the  aiding 
system.  However,  if  the  pilot  does  not  want  the  aid  to  execute  a  given  task 
(e.g.,  if  the  pilot  is  aware  that  a  target  is  using  a  tactic  not  known  to  the 
aiding  system),  he  should  be  able  to  regain  control  of  that  task.  This 
suggests  that  both  operator  and  system  control  of  task  allocation  will  help 
optimize  system  performance. 

Empirical  Support:  This  guideline  is  supported  by  the  CAS1  study  in  the 
Morrison  et  al.  paper. 

4.2.1  .f  Dynamic  allocation  of  tasks  should  be  employed  to  insure  that  a  given  task 
is  optimally  assigned  to  the  agent  that  is  better  able  to  perform  that  task. 
Dynamic  allocation  uses  the  system’s  resources  more  effectively  than 
static  allocation  (as  is  the  case  in  Fitts'  list)  (Krobusek  et  al.  1 989). 

Explanation:  Task  allocation  is  static  when  it  is  based  on  the  functional 
abilities  of  the  human  operator  or  machine  as  determined  by  the  Fitts  List. 
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Static  allocation  does  not  account  for  changes  in  either  humans'  abilities 
or  environmental  conditions  over  time.  Dynamic  allocation  Is,  however, 
based  variable  conditions  and  ensures  that  a  given  task  is  optimally 
assigned  to  the  agent  that  is  better  able  to  perform  that  task  at  that  point  in 
time. 

Empirical  Support:  Though  widely  accepted  in  the  adaptive  aiding 
domain,  this  guideline  requires  further  support. 

4.2.1  .g  Problem  solving  should  be  allocated  to  the  agent  that  is  likely  to  have  the 
better  solution  to  the  problem.  The  implies  that  the  decision  maker  must  be 
able  to  discriminate  situations  on  the  basis  of  who  (operator  or  aid)  is  more 
likely  to  be  correct  (Lehner  et  al.  1989). 

Explanation:  This  guideline  is  similar  to  Guideline  4.2.1. f,  but  carries  the 
additional  implication  that  if  tasks  are  dynamically  allocated  by  the 
decision  maker  (the  human  operator),  he  must  be  able  to  tell  whether  he  or 
the  aid  will  provide  the  better  problem  solution.  A  point  to  consider  here  is 
that  operator  performance  is  better  when  the  human  feels  in  control  of  the 
system,  even  when  the  aid  is  not  active  (Morris  and  Rouse,  1986; 

Guideline  4.2. 1.c).  Therefore,  system  performance  may  be  optimal  when 
the  human  controls  task  allocation. 

Empirical  Support:  The  Lehner  et  al.  study  supports  this  guideline  as  does 
Morris  and  Rouse  (1 986). 

4.2.1  .h  The  amount  of  stimulus  information  classified  per  unit  time  depends  on: 

size  of  stimulus  class,  experience  level  of  the  operator,  speed/accuracy  of 
operator,  and  the  stimulus-response  compatibility  of  the  displays  and 
controls  (Barnes,  1981). 

Explanation:  The  way  information  is  displayed  affects  the  ability  of  the 
human  operator  to  classify  that  stimulus  information  and  to  perform  tasks 
assigned  to  him.  Interfaces  that  display  aiding  information  should 
therefore  be  designed  to  optimize  the  speed  with  which  an  operator  can 
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classify  that  information  and  subsequently  make  decisions.  Barnes  also 
provides  guidelines  for  information  display. 

Empirical  Support:  This  guideline  is  directly  supported  by  Barnes'  study. 

4.2.1  .i  The  pilot  must  be  trained  to  configure  the  system  both  before  the  mission 
and  during  the  mission.  Loss  of  pilot  skill  may  occur  due  to  automation. 
Intermittent  retraining  and  embedded  training  should  be  considered  in  aid 
design  (Krobusek  et  al.  1989). 

•Explanation:  Training  requirements  increase  with  the  use  of  aiding 
systems  because  pilots  must  configure  automated  systems  both  before 
and  during  the  mission.  It  is  also  possible  for  extensive  automation  to 
degrade  pilot  skill  over  time  because  reliable  automation  places  fewer  task 
performance  demands  on  pilots.  If  pilots  are  not  able  to  practice  flight- 
related  tasks,  they  will  ultimately  lose  their  ability  to  perform  those  tasks 
and  overall  system  performance  will  degrade.  The  solution  to  these 
training  issues  is  the  implementation  of  intermittent  and  embedded  training 
within  the  aid. 

Empirical  Support: 

4.2.1 .]  Designers  are  not  interested  in  tradeoffs  between  costs  of  communicating 
vs.  costs  of  aiding;  the  necessary  level  of  aid  tailorability;  and  number  and 
applicability  of  interface  /  aiding  models  available  (Andes  and  Rouse, 

1991). 

Explanation:  In  a  study  conducted  with  aiding  system  designers,  Andes 
and  Rouse  found  that  designers  do  not  believe  that  the  factors  listed  above 
affect  system  performance  as  much  as:  the  method  of  task  allocation 
utilized,  the  availability  of  aiding,  the  timeliness  of  pilot  response,  and  pilot 
training  requirements.  Designers  place  higher  value  on  this  information  as 
it  affects  system  design. 

Empirical  Support:  Additional  research  must  be  conducted  to  determine 
whether  designs  that  do  not  incorporate  aiding  tradeoff,  aid  tailorability 
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level,  and  interface  /  aiding  model  Information  are  of  a  poorer  quality  than 
systems  that  do  Incorporate  this  Information. 

4.2.2  Implications  on  Workload 

Adaptation  of  the  operator  or  aid  can  affect  the  workload  that  either  of  these 
agents  experience.  In  cases  of  static  adaptation,  workload  may  be  initially  increased 
due  to  the  reallocation  of  tasks  during  the  process  of  adaptation.  In  dynamic 
adaptation,  the  process  of  adaptation  is  continuous  for  the  agent  of  adaptation. 
Human  performance  characteristics,  relative  abilities,  and  context  must  be 
considered  when  analyzing  the  effects  of  agent  of  adaptation  on  resulting  workload. 

Guidelines 

4.2.2. a  Long  lists  of  information,  tasks,  etc.  should  be  stored  and  prioritized  by  the 

aid  to  minimize  the  number  of  decision  alternatives  and  reduce  the  visual 
processing  load  of  the  human  operator  (Barnes,  1981). 

Explanation:  In  multi-task  environments,  the  aid  should  prioritize  and  store 
tasks  to  be  performed.  This  will  minimize  workload  in  terms  of  both 
decision  making  and  visual  processing  requirements. 

Empirical  Support:  This  guideline  has  been  supported  by  Greenstein  and 
Revesman  (1986),  and  Sage  and  White  (1984). 

4.2.2.  b  Adaptive  aiding  and  decision  utility  techniques  are  particularly  applicable 

to  situations  where  humans  must  solve  complex  problems  that  do  not  yield 
to  analysis  or  strict  adherence  to  doctrine  or  standard  operating 
procedures  (Weisbrod,  Davis  and  Freedy,  1977). 

Explanation:  Adaptive  aiding  and  decision  utility  assessment  are  well- 
suited  for  complex  problem  situations.  The  flexible  nature  of  adaptive 
systems  allows  the  aid  to  address  tasks  that  it  would  perform  best,  and 
allows  the  human  to  address  other  tasks.  Adaptive  aids  minimize  operator 
workload  because  the  aid  can  perform  repetitive  and  boring  tasks,  leaving 
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the  human  to  perform  complex  tasks  that  the  aid  might  not  be  capable  of 
performing. 

Empirical  Support:  Support  for  this  guideline  is  provided  in  this  study. 

4.2.2. C  Consider  operator  fatigue  in  the  design  of  an  aiding  system.  Increased 

aiding  may  be  necessary;  task  demands  that  were  acceptable  at  the 
beginning  of  a  task  may  impose  high  workload  later,  when  the  operator 
becomes  fatigued  (Morris,  Rouse,  Ward  and  Frey,  1984). 

Explanation:  Operator  fatigue  is  directly  related  to  the  concept  of  workload. 
When  the  operator  experiences  fatigue,  he  perceives  increases  in 
workload  even  if  task  demands  do  not  change.  Thus,  task  demands  that 
were  originally  acceptable  will  later  impose  high  workload  and  will  require 
increases  in  aiding.  This  guideline  suggests  that  designers  account  for 
increases  in  operator  fatigue  and  workload  in  the  design  of  aiding 
applications. 

Empirical  Support:  Numerous  studies  on  vigilance  support  this  guideline, 
but  it  has  not  been  validated  within  the  adaptive  aiding  domain. 

4.2.3  Implications  on  Operator  Acceptance 

Operator  acceptance  of  an  aid  may  be  strongly  affected  by  whether  the 
operator  or  the  aid  does  the  adapting.  Operator  adaptation  may  decrease 
acceptance  because  the  operator  has  the  added  responsibility  to  under  what 
conditions  to  adapt.  Alternatively,  the  aid  that  adapts  without  notifying  the  operator 
may  also  decrease  acceptance  of  the  aid.  Several  characteristics  about  the  aid  must 
therefore  be  considered  to  ensure  user  acceptance. 

Guidelines 

4.2.3. a  An  aid  may  have  to  be  much  better  at  executing  a  system  task  than  users 

In  order  to  be  accepted  by  the  user  population  (Rouse,  1988). 
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Explanation:  Users  often  perceive  their  own  performance  as  better  than  it 
actually  Is.  Therefore,  for  an  aid  to  be  acceptable,  it  will  have  to  perform 
better  than  the  users'  perceptions  of  their  own  performance. 

Empirical  Support:  This  guideline  is  supported  by  Morris,  Rouse  and  Ward 
(1985). 

4.2.3. b  The  pilot  will  not  accept  automation  unless  he  approves  of  the  operational 

relationship  between  the  aid  and  himself  (Krobusek  et  al.  1989). 

Explanation:  In  order  for  the  pilot  to  accept  an  aid,  he  must  feel.as  if  he  as 
a  certain  degree  of  control  over  the  aid. 

Empirical  Support:  An  Interesting  empirical  discussion  and  validation  of 
this  guideline  can  be  found  in  Morris  and  Rouse  (1986),  and  Morris,  Rouse 
and  Ward  (1985). 

4.2.3. C  An  adaptive  aid  must  perform  tasks  as  accurately  as  the  operator.  To  do 

this,  the  aid  must  unobtrusively  monitor  and  record  actual  operator 
performance  (Andes,  1987). 

Explanation:  The  operator  will  not  accept  an  aid  if  it  interferes  with  task 
performance.  Neither  will  he  accept  an  aid  if  it  Is  inaccurate.  Therefore,  an 
aid  must  monitor  the  operator  without  disturbing  him.  In  an  effort  to 
achieve  unobtrusive  monitoring,  Morris  and  Rouse  (1986)  have  developed 
several  techniques.  Including  Mean  Squared  Error  (MSE)  analysis, 
workload  modeling,  and  queueing  models  of  performance. 

Empirical  Support:  This  guideline  was  validated  in  the  Morris  and  Rouse 
(1986)  study.  In  addition,  Chu  and  Rouse  (1979)  show  the  application  of 
some  of  the  above  techniques. 

4.2.3. d  Minimize  the  number  of  system  variables  required  through  operator 

tailoring  of  the  system.  Operator  tailoring  involves  personally  tailoring  the 
aid  to  the  pilot  depending  on  the  specific  mission,  mission  tasks  required, 
and  pilot  preference  (Lind,  1989). 
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Explanation:  Operator  tailoring  is  another  way  to  approach  the  operator 
acceptance  issue.  In  operator  tailoring,  the  pilot  is  required  to  attend  to  a 
minimum  number  of  system  variables,  and  this  minimizes  pilot  workload. 
Furthermore,  the  aiding  system  is  tailored  to  each  pilot,  taking  into  account 
the  type  of  mission,  required  mission  tasks,  and  pilot  preferences.  It  is 
thought  that  acceptance  will  be  achieved  if  pilots  are  allowed  to  specify 
how  the  aiding  system  will  function. 

Empirical  Support:  This  guideline  was  supported  in  the  Lind  (1989)  study, 
where  pilots  gave  favorable  reviews  to.  operator  tailoring  during  evaluation. 

4.2.4  implications  on  Situation  Assessment 

If  either  the  operator  or  the  aid  are  engaged  in  adaptation  activities,  the  ability 
of  either  agent  to  assess  situations  will  be  affected.  This  ability  will  most  likely  be 
impaired  due  to  the  decreased  processing  capacity  of  the  operator  and  the  aid 
during  adaptation  activities.  With  less  room  to  process  information,  situational 
information  is  less  likely  to  be  processed  and  used  in  an  assessment. 

Guidelines 

4.2.4.a  These  factors  affect  the  quality  of  the  human's  decisions:  human's  attitude 
towards  aid;  human's  perception  of  performance  criteria;  the  situation;  his 
own  performance;  and  the  computer's  performance  (Morris  and  Rouse, 
1985). 

Explanation:  Situational  awareness  is  attained  when  the  human  makes 
judgments  about  the  state  of  the  current  situation.  Since  situational 
awareness  is  part  of  the  decision  making  process,  it  is  also  affected  by  the 
above  factors.  Therefore,  the  attitude  toward  the  aid,  the  perception  of 
performance  criteria,  actual  performance,  and  computer  performance  all 
affect  the  ability  of  the  human  to  assess  situations. 

Empirical  Support:  Although  the  effects  of  these  factors  have  not  been 
validated  in  the  context  of  aiding  systems,  it  is  believed  that  these  factors 
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do  Indeed  interact  with  each  other,  and  consequently  affect  overall  system 
performance. 

4.2.4. b  The  operator  should  perform  system  monitoring  tasks  some  of  the  time  by 

employing  adaptive  automation,  since  the  aid  would  not  always  be  on. 
This  would  reduce  performance  decrements  related  to  vigilance  because 
the  operator  is  still  active  in  controlling  the  system  (Mom’son  et  al.  1990). 

Explanation:  System  monitoring  tasks  are  especially  susceptible  to 
performance  decrements  related  to  vigilance  (Morrison,  Gluckman  and 
Deaton,  1990).  Morrison  et  al.  suggest  using  adaptive  automation  so  that 
the  pilot  can  maintain  situational  awareness  by  performing  monitoring 
tasks  periodically.  It  is  believed  that  periodically  aided  monitoring  may 
benefit  overall  system  performance  because  the  operator  would  be  able  to 
maintain  situational  awareness  without  experiencing  performance 
decrements  related  to  vigilance. 

Empirical  Support:  Additional  research  must  be  done  in  the  context  of 
aiding  to  insure  that  adaptive  automation  reliably  minimizes  vigilance 
related  performance  decrements. 

4.2.4. C  Make  it  very  clear  whether  the  human  or  computer  is  supposed  to  perform 

a  particular  task  at  a  specific  time  and  also  provide  a  means  for  changing 
the  allocation  (Rouse,  1988). 

Explanation:  The  nature  of  task  allocation  also  affects  the  ability  for  the 
human  to  assess  a  given  situation.  If  too  many  tasks  are  allocated  to  the 
human  or  if  he  is  unsure  about  whether  or  when  to  perform  tasks,  he  will 
not  be  able  to  maintain  an  awareness  of  the  situation.  This  conclusion  is 
drawn  from  the  work  of  Lehner  et  al.  (1987),  who  advise  to  provide  a 
means  to  avoid  user  confusion  in  reaction  to  aid-initiated  adaptation. 
Krobusek,  Boys  and  Palko  (1989)  also  advise  against  the  application  of  a 
wide-spectrum  pilot  aid  in  order  to  preempt  confusion  resulting  from 
unclearly  specified  task  allocation.  Finally,  Chu,  and  Rouse  (1979)  advise 
to  implement  a  method  by  which  the  user  can  preempt  adaptation,  or  "take 
back"  a  task  that  was  previously  allocated  to  the  aiding  system. 
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Empirical  Support: 

4.2:  Section  Summary 

This  section  discussed  the  implications  of  whether  the  operator  or  the  aid  is 
the  agent  of  adaptation  in  an  aiding  system.  The  guidelines  in  Section  4.2  suggest 
that  the  human  is  the  default  initiator  of  aiding.  Further,  research  shows  that  the 
presence  of  an  aid  positively  influences  human/system  performance,  that  the  aid 
should  initiate  tasks  that  pilots  are  unlikely  to  perform,  and  that  pilots  should  be 
sufficiently  trained  in  system  configuration.  These  guidelines  also  promote  dynamic 
task  allocation,  and  emphasize  the  Importance  of  high  aid  reliability,  operator 
tailoring,  and  the  minimization  of  tasks  which  induce  operator  fatigue.  These 
guidelines  suggest  that  designers  should;  a)  implement  aiding  systems  that  are 
explicit  about  task  allocation,  and  b)  optimize  the  information  processing  abilities  of 
the  operator. 


4.3  Guidelines  for  "When  Does  Adaptation  Occur?" 

Adaptation  may  occur  before  system  operation,  or  in  response  to  changing 
system  conditions.  Pre-operation  adaptation  accounts  for  off-line  or  static  types  of 
information  that  do  not  change  throughout  system  operation.  Once  system  operation 
begins,  however,  adaptation  occurs  in  response  to  on-line  or  dynamic  information 
that  changes  with  varying  system  conditions.  The  timing  of  adaptation  may  affect  all 
facets  of  system  operation.  Timing  of  adaptation  can  depend  on  various 
"intervention  thresholds."  Different  approaches  to  determining  the  aiding  triggers 
have  been  investigated;  the  results  and  Implications  are  examined  in  this  section. 

4.3.1  Implications  on  Performance 

A  few  different  methods  have  been  used  to  determine  when  system 
performance  will  benefit  from  the  use  of  adaptive  aiding.  Timing  of  intervention  is  of 
paramount  importance  to  insure  that  system  performance  is  not  further  degraded  in 
the  transition  from  unaided  to  aided  performance.  These  implications  are  discussed 
in  more  detail  below. 
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Guidelines 

4.3.1  .a  Use  mission-mode  to  determine  suite  of  functions  that  should  be 
automated  during  specific  segments  of  the  mission  (Lind,  1989). 

Explanation:  Before  the  mission,  the  pilot  may  determine  for  which  parts  or 
modes  of  the  mission  particular  functions  will  be  automated.  This  is  called 
mission-mode  tailoring.  This  method  of  automation  Is  useful  when  it  is  not 
possible  to  dynamically  apply  automation  in  response  to  changing  systems 
demands. 

Empirical  Support:  This  guideline  is  validated  in  the  Lind  (1989)  study. 

4.3.1.  b  Queueing  model  parameters  of  operator  workload  models,  specifically 

event  arrival  and  service  time,  are  useful  In  a  coarse  prediction  of  levels  of 
task  loading  in  multi-task  situations.  Utility  is  limited,  however.  Use  when 
accurate  methods  are  not  available  (Chu  and  Rouse,  1979). 

Explanation:  Chu  and  Rouse  have  found  that  event  arrival  and  service 
time  are  useful  in  predicting  when  optimal  levels  of  workload  are 
exceeded.  Therefore,  they  may  be  used  to  determine  when  operators 
should  be  aided  in  order  to  maintain  acceptable  levels  of  performance. 
Since  these  models  have  only  coarse  predictive  abilities,  they  should  be 
used  when  more  accurate  methods  of  workload  prediction  are  not 
available. 

Empirical  Support:  This  guideline  was  validated  in  the  study. 

4.3.1.  c  Modified  Petri  nets  can  be  used  to  model  operator  related  processes  or 

activities,  in  terms  of  both  operator  information  I/O  and  interaction  I/O. 

These  Petri  nets  are  used  to  represent  "when"  or  control  knowledge 
associated  with  allocation  and  viable  automation  (Madni,  1988). 

Explanation:  Modified  Petri  nets  are  a  useful  and  flexible  way  to  model 
operator  processes  and  activities.  They  are  capable  of  modeling  operator 
information  input  and  output,  and  interaction  input  and  output. 
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Furthermore,  modified  Petri  nets  can  represent  knowledge  about  when  to 
allocate  tasks  and  when  to  automate  tasks. 

Empirical  Support: 

4.3.1. d  An  aiding  system  should  have  mechanisms  to  store  task  procedures, 
threshold  values,  and  estimates  of  aiding  success  (context  specific,  if 
possible).  This  data  can  be  used  to  evaluate  current  performance  and  use 
feedback  to  improve  future  aid  performance  (Andes,  1987). 

Explanation;  The  availability  of  this  information  is  critical  to  the  aid  during 
system  performance  evaluations  and  in  using  feedback  from  evaluations  to 
improve  future  aiding  performance. 

Empirical  Support: 

4.3.1  .e  Adaptation  should  occur  when  the  user  knows  exactly  how  to 
reach  task  objectives;  when  the  user  generally  knows  how  to  reach 
objectiveSrbut  not  efficiently;  when  the  user  knows  little  about  how  to  reach 
.objectives;  and  when  user  wants  peripheral  information  (Tyler  and  Treu, 
1989). 

Explanation:  Tyler  and  Treu  believe  that  decision  system  performance  will 
be  maximized  when  adaptive  aiding  occurs  In  the  situations  listed  above. 
Adaptation  situations  have  also  been  discussed  in  Guideline  4.3.1.f 
(Andes,  1987)  with  a  focus  on  a  different  application  domain. 

Empirical  Support: 

4.3.1  .f  Adaptation  should  occur  when:  an  undetected  operator  error  is  committed; 
the  operator  is  too  busy  to  take  on  another  task;  allocation  is  desired  by  the 
operator;  the  operator  is  functionally  impaired;  and  /  or  critical  (life- 
threatening)  situations  exist  (Andes,  1987).  For  application  in  dynamic 
systems  control. 
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Explanation:  Ancles  explains  that  performance  of  the  Pilot’s  Associate 
system  (a  dynamic,  tactical  aircraft  automation  program)  will  be  maximized 
when  adaptive  aiding  is  applied  during  the  situations  listed  above. 

Empirical  Support:  Although  these  specifications  are  consistent  with  the 
literature  thus  far,  they  have  not  been  validated  empirically. 

4.3.1  .g  It  is  desirable  to  aid  situations  where  task  difficulty  will  increase  because 

the  resource  demands  of  responding  are  increasing.  Human  operators  do 
not  subjectively  perceive  this  situation  as  more  difficult,  even  though 
performance  deteriorates  (Derrick,  1988). 

Explanation:  Aid  should  be  applied  when  increases  in  task  difficulty  are 
predicted  because  of  concomitant  increases  in  the  resource  demands  of 
making  a  response. 

Empirical  Support:  This  guideline  has  behavioral  support.  Although 
humans  do  not  always  perceive  increases  in  task  difficulty,  performance 
deteriorates  nonetheless  (Wickens  and  Yeh,  1983).  This  suggests  that  aid 
should  be  automatically  Invoked  when  increases  in  task  difficulty  are 
predicted. 

4.3.2  Implications  on  Workload 

Aiding  that  adapts  to  changing  system  conditions  may  significantly  affect  the 
operator’s  workload  profile.  Specifically,  such  aiding  may  improve  the  individual 
operator's  ability  to  perform  tasks  allocated  to  him  due  to  the  decreased  overall 
workload.  It  is  critical  to  consider,  therefore,  when  adaptation  would  best  be  initiated 
in  terms  of  workload. 

Guidelines 

4.3.2.a  Use  automation  under  conditions  of  time  pressure  to  help  the  pilot  cope 
with  an  increasingly  complex  environment  (Parasuraman  et  al.  1990). 
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Explanation:  Time  pressured  situations  increase  operators’  perceptions  of 
workload.  Automation  of  tasks  that  are  suitable  for  automation  can 
decrease  operator  workload  while  increasing  mission  safety. 

Empirical  Support:  This  guideline  is  supported  by  the  fact  that  automated 
systems  are  more  reliable  than  humans. 

4.3.2.b  Decision  errors  occur  when  the  decision  maker  generates  a  wrong 
individual  response.  This  type  of  error  occurs  at  medium  and  high 
information  input  rates  (Sen,  1984). 

Explanation:  Workload  is  considered  high  at  medium  and  high  information 
input  rates.  Sen  has  shown  that  error  behavior  occurs  in  high  workload 
situations  when  decision  makers  generate  wrong  responses  for  given 
conditions. 


Empirical  Support:  Miller's  (1969)  work  provided  the  basis  for  Sen’s 
(1984)  study,  both  of  which  validate  this  guideline. 


4.3.2.C 


The  one  main  difference  between  Idea!  vs  Human  decision  maker  is  that" 


the  human  knows  when  he  is  being  overloaded  by  noticing  increases  in 
the  number  of  input  characteristics  he  must  ignore  to  make  a  timely 
decision  (Sen,  1984). 


Explanation:  The  human  perceives  increases  in  workload  by  noticing 
(increases  in  the  number  of  input  characteristics  that  he  must  ignore  in 
order  to  make  a  timely  decision.  The  mathematically  ideal  decision  maker 
also  ignores  input  characteristics  to  avoid  information  overload,  but  this  set 
of  characteristics  is  likely  to  be  a  smaller  subset  than  that  of  the  human. 

The  optimal  point  of  aiding  then,  is  the  point  at  which  the  human  ignores  a 
critical  number  of  input  characteristics. 

Empirical  Support:  This  guideline  is  validated  by  this  study. 

4.3.2.d  When  using  workload  as  a  dimension  for  intervention,  task  characteristics 
used  to  predict  workload  must  be  augmented  with  the  operator’s  perceived 
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workload  over  tasks.  Based  on  Wickens'  Multiple  Resource  Theory, 
Derrick  found  that  the  processing  stages  dimension  is  the  most  important 
of  those  available  (Derrick,  1988). 

Explanation:  Perceived  workload  may  be  higher  than  levels  of  workload 
estimated  from  task  characteristics.  When  predicting  workload  then,  task 
characteristics  must  be  amended  so  that  estimated  workload  will 
correspond  with  the  operator's  perceived  workload.  Derrick  also  found 
that  the  processing  stages  dimension  of  Wickens'  (1984)  Multiple 
Resource  Theory  is  most  important  in  predicting  workload.  This  is  critical 
in  finding  at  what  point  during  information  processing  aiding  would  be 
most  beneficial. 

Empirical  Support:  These  findings  also  apply  to  the  use  of  implicit 
communication  for  Invoking  aid-initiated  intervention,  and  are  additionally 
supported  by  Wickens  and  Yeh  (1983). 

4.3.2.e  Tailored  logic  can  be  employed  to  alter  the  system  aiding  menus  to  reflect 
the  needs  of  the  current  tactical  environment.  This  approach  reduces  pilot 
workload  by  dynamically  customizing  the  menus.  It  can  be  triggered  by 
mission-mode,  situation  assessment,  etc.  (Barnes,  1985). 

Explanation:  Tailored  logic  is  an  example  of  partitioning  tasks  in  the 
cockpit,  where  tasks  and  parts  of  tasks  are  shared  between  the  operator 
and  the  aid.  Pilot  workload  can  be  reduced  by  dynamically  customizing 
menus  through  tailored  logic:  the  menus  offer  only  those  options  that  are 
relevant  to  the  current  mission  environment.  Howard,  Hammer  and 
Geddes  (1988)  have  developed  tailored  logic  within  the  Pilot’s  Associate 
system. 

Empirical  Support: 

4.3.3  Implications  on  Operator  Acceptance 

The  timing  of  adaptation  may  have  critical  impact  on  an  operator's  acceptance 
of  an  aiding  system.  Operator  acceptance  may  be  negatively  affected  if  the  aiding 
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occurs  unexpectedly,  or  at  a  time  when  the  operator  may  not  desire  the  aiding. 

Some  guidelines  addressing  this  issue  are  enumerated  below. 

Guidelines 

4.3.3. a  Ensure  that  user-initiated  adaptation  is  possible  and  appropriately 

supported,  even  if  aid-initiated  adaptation  is  the  norm  (Rouse,  1988). 

Explanation:  Operator  acceptance  of  any  aiding  system  hinges  on 
whether  or  not  the  operator  feels  in  control  of  the  system  (Rouse,  1988). 
Since  the  operator  is  ultimately  responsible  for  system  behavior,  he  should 
be  able  to  exert  direct  control  in  the  system. 

Empirical  Support:  This  guideline  is  supported  by  studies  which  show  that 
operators  prefer  to  be  In  control  of  the  system.  These  studies  include 
Morris  and  Rouse  (1986),  and  Morris,  Rouse,  and  Ward  (1985), 

4.3.3. b  The  pilot  must  always  be  in  control  of  the  aircraft,  must  be  allowed  to  define 

and  change  his  role  in  the  system,  and  be  able  to  override  any  aid  activity 
(Krobusek  et  al.  1989). 

Explanation:  The  levels  of  autonomy  (LOA)  philosophy  (Krobusek  et  al. 
1989)  is  that  the  pilot  is  able  to  specify  when  and  how  the  aid  will  be  active. 
This  paradigm  is  based  on  the  assumption  that  operator  acceptance  can 
be  achieved  by  allowing  the  pilot  to  control  the  system  and  change  its 
configuration  at  any  time.  Forms  of  this  guideline  pervade  the  adaptive 
aiding  literature.  It  is  specifically  supported  In  Rouse,  Geddes  and  Curry 
(1987). 

Empirical  Support: 

4.3.3. C  Use  workload  measurement  technology  (e.g.,  secondary  task,  subjective 

measures,  psychophysical  indices,  etc.)  to  indicate  exactly  what 
component  of  mental  workload  is  overloaded,  otherwise  the  aid  may  off¬ 
load  tasks  from  the  pilot  when  his  resources  are  not  overloaded 
(Parasuraman  et  al.  1990). 
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Explanation:  This  guideline  is  important  to  insuring  that  the  aid  is  invoked 
at  appropriate  times.  If  an  aid  cannot  determine  exactly  which  components 
of  mental  workload  are  overloaded,  tasks  may  be  off-loaded  from  the  pilot 
when  it  is  not  necessary.  This  may  result  in  pilot  confusion  and/or 
significant  increases  in  workload,  and  in  the  long-run,  operator  rejection  of 
the  system.  Workload  measurement  technology  allows  the  aid  to  predict 
the  overload  of  specific  mental  components.  Note  that  this  guideline  also 
has  implications  for  workload. 

Empirical  Support:  Empirical  support  for  using  workload  measurement 
technology  to  foster  user  acceptance  can  be  found  in  Wickens  (1984)  and 
Lehner,  Cohen,  Mullin,  Thompson,  and  Laskey  (1987). 

4.3.3. d  Avoid  excessive  task  responsibility  trading  by  allowing  the  user  to  off-load 

and  recapture  tasks  to  and  from  the  aid  as  desired  (Rouse,  1988). 

Explanation:  The  "hot  potato"  phenomenon  occurs  when  the  aid  initiates 
excessive  task  responsibility  trading  between  the  operator  and  the  aid. 

This  can  be  avoided  by  allowing  the  user  to  give  up  tasks  to  the  aid  and 
get  back  from  the  aid  as  desired.  Although  this  may  allow  the  user  to  feel 
in  charge  of  the  overall  system  and  may  facilitate  operator  acceptance  of 
the  aid,  it  may  result  in  performance  hysteresis,  a  problem  addressed  in 
Guideline  4.3.3.e. 

Empirical  Support: 

4.3.3. e  Performance  hysteresis  should  be  avoided.  Performance  hysteresis 

occurs  with  cyclic  Introduction  and  removal  of  aiding  where  operator 
performance  approaches  local  maxima  although  the  general  performance 
trend  Is  downward  (Andes,  1990).  See  4.3.3.d. 

Explanation:  Performance  hysteresis  is  ultimately  detrimental  to  overall 
system  performance.  Although  operator  performance  may  approach  local 
maxima  between  instances  of  task  off-load  and  recapture,  operator 
performance  generally  degrades.  By  accounting  for  the  hot  potato 
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phenomenon  and  performance  hysteresis,  the  designer  can  move  toward 
a  more  symbiotic  operator-aid  system  -  one  that  operators  will  want  to  use. 

Empirical  Support: 

4.3.4  Implications  on  Situation  Assessment 

The  operator's  perception  of  when  adaptation  should  occur  is  important  in 
maintaining  situation  assessment  of  both  aid  functionality  and  overall  situation 
assessment.  Although  this  is  an  important  implication,  only  a  small  number  of 
guidelines  were  found. 

Guidelines 

4.3.4.a  Familiarity  with  the  current  situation  and  perceived  normal  functioning  of 
the  aid  are  two  main  factors  affecting  the  human  operator's  decision  of 
whether  to  use  an  aid.  The  operator  should  possess  sufficient  information 
to  successfully  address  the  factors  affecting  the  system's  current  status 
(Morris,  Rouse  and  Ward,  1985). 

Explanation:  So  far,  there  is  only  one  guideline  that  is  associated  with 
situation  assessment  in  deciding  when  to  apply  aiding.  Morris,  Rouse  and 
Ward  have  found  that  the  human  operator's  decision  to  use  an  aid  is 
affected  by  situational  familiarity  and  perceived  normal  functioning  of  the 
aid.  Thus,  the  operator  must  have  a  certain  degree  of  situation  awareness 
in  order  to  efficiently  decide  whether  to  invoke  the  aid.  Situation 
awareness  will  also  enable  the  operator  to  ascertain  whether  the  aid  is 
capable  of  handling  tasks  in  unfamiliar  contexts. 

Empirical  Support: 

4.3:  Section  Summary 

This  section  discussed  issues  pertaining  to  to  the  initiation  of  the  aid  in  an 
aiding  system.  These  guidelines  suggest  that  the  invocation  of  an  aid  should  be 
based  on  mission  mode  analyses,  models  of  operator  cognitive  processes,  the 
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amount  of  Information  available  for  decision  making,  the  level  of  operator  workload, 
and  the  operator’s  physical  state.  Related  research  shows  that  an  aid  should  be 
invoked  during  Increases  in  both  time-critical  tasks  and  the  number  of  system 
variables  that  must  be  monitored,  but  that  excessive  task-trading  should  be  avoided. 
As  emphasized  in  Sections  4.1  and  4.2,  user-initiated  adaptation  should  be  possible 
as  well.  These  guidelines  suggest  that  designers  should:  a)  implement  aiding 
systems  that  give  operators  the  Impression  that  they  are  actively  controlling  the 
system,  and  b)  emphasize  pilot  training  in  order  to  increase  operator  situation 
assessment  abilities. 


4.4  What  Methods  of  Adaptation  Apply? 

According  to  Rouse  and  Rouse  (1 983),  there  are  three  primary  ways  to 
implement  adaptation  in  an  aiding  system:  allocation  (designating  whether  the 
operator  or  the  aid  will  perform  a  task),  partitioning  (the  sharing  of  tasks  between  the 
operator  and  the  aid),  and  transformation  (changing  a  task  to  make  it  easier  to 
perform).  These  methods  are  discussed  with  regard  to  their  effects  on  the  aiding 
system.  Most  of  the  other  paradigms  (e.g.,  function  aiding  [Fitts,  1951 ;  Lind,  1989], 
LOA  [Krobusek  et  al.  1989],  etc.)  fall  into  the  allocation  type  of  adaptation  in  the 
adaptive  aiding  concept. 

4.4.1  Implications  on  Performance 

System  performance  depends  on  the  designers'  abilities  to  correctly  account 
for  all  necessary  variables  during  system  design.  It  is  also  affected  by  the  abilities  of 
the  operators  to  efficiently  invoke  aiding  when  it  is  their  decision  to  do  so.  The 
abilities  of  designers  and  operators  are  influenced  by  the  availability  of  the 
information  required  to  make  design  or  operation  decisions.  This  section  presents 
research  about  the  issues  of  system,  operator,  and  designer  performance  as  they 
relate  to  methods  of  adaptation. 

Guidelines 

4.4.1, a  Designers  are  particularly  interested  in  the  availability  of  technology  to 
support  an  aiding  implementation  (Andes  and  Rouse,  1991). 
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Explanation:  Apparently,  designers  are  confident  in  what  they  want  to  do, 
but  require  more  information  about  the  methods  for  implementing  aiding 
systems.  Performance  of  designers  (in  terms  of  the  types  of  aiding  systems 
that  are  ultimately  developed)  will  be  largely  affected  by  the  availability  of 
technology  to  implement  aiding  systems  with  particular  methodologies. 

Empirical  Support:  Although  this  was  a  pilot  study,  some  support  for  this 
guideline  was  provided. 

4.4.1. b  A  complete  systems  engineering  approach  is  imperative  to  aiding  systems 

design  due  to  the  complexity  of  the  human-machine  interface.  It  requires 
complete  user  control  and  information  requirements,  function  allocation, 
and  consideration  of  the  capabilities  and  constraints  of  the  system 
(Parasuraman  et  al.  1990). 

Explanation:  The  systems  engineering  approach  incorporates 
assumptions  about  the  capabilities  and  constraints  of  the  system  with 
information  requirements  analyses,  function  analyses,  and  information  and 
control  requirements  analyses,  it  Is  believed  this  approach  will  provide 
designers  with  a  method  of  accounting  for  all  of  the  necessary  information 
in  the  design  of  aiding  systems. 

Empirical  Support:  The  systems  engineering  approach  has  been 
proposed  but  has  not  been  validated  by  an  empirical  evaluation  of  an 
aiding  system  design  methodology. 

4.4.1  .c  Critical-event  logic  (see  Glossary  of  Terms)  can  be  used  to  automate 

systems  functions  when  mission  critical  events  occur.  Automated  function 
suites  can  be  based  on:  emergency  logic,  executive  logic,  and  automated 
display  logic  (Parasuraman  et  al.  1990). 

Explanation:  High  levels  of  system  performance  may  be  achieved  by 
Implementing  critical  event  logic  when  mission-critical  events  occur. 
Depending  on  the  situational  context,  automation  may  be  Invoked  without 
pilot  approval  when  the  system  matches  pre-defined  events  with  the 
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current  events.  This  approach  is  conceptually  simple,  however,  since  it  is 
insensitive  to  actual  pilot  performance  or  workload,  it  may  not  be  the  best 
way  to  insure  acceptable  system  performance. 

Empirical  Support:  Barnes  (1981)  considers  Information  requirements  in 
the  face  of  critical  events,  but  validation  of  this  guideline  has  yet  to  be 
achieved. 

4.4.1  .d  The  LOA  (levels  of  autonomy)  paradigm  may  be  used  to  circumvent 

problems  related  to  allocation  in  systems  with  multiple  configurations.  LOA 
defines  a  set  of  aiding  configurations,  each  of  which  specifies  a  degree  of 
automation  per  system  subfunction  (Krobusek  et  al.  1989). 

Explanation:  A  system  developed  within  the  LOA  approach  can  define  a 
set  of  possible  aiding  configurations,  differing  in  the  degree  of  automation 
per  system  subfunction.  The  pilot  sets  the  level  of  system  autonomy 
required  per  task  as  determined  by  mission  planning  task  analysis  or  in¬ 
flight  needs.  The  system  then  executes  the  clearly  defined  tasks  that  it  has 
been  assigned.  The  LOA  method  of  aiding  applies  only  to  task  allocation 
problems.  It  is  therefore  not  useful  for  error  compensation  or  multi-mode, 
aid-initiated  aiding  situations. 

Empirical  Support:  The  LOA  methodology  has  not  been  validated. 

4.4.1. e  LOA  (Levels  of  Automation)  configurations  include:  no  intervention;  aid 

automatically  executes  tasks;  aid  reminds  pilot  of  performed  tasks;  aid 
prompts  pilot  about  important  tasks;  aid  performs  tasks  when  pre-specified 
conditions  are  met  (Krobusek  et  al.  1989). 

Explanation:  The  five  possible  LOA  aiding  system  configurations  are  listed 
above.  They  have  been  developed  on  the  assumption  that  task  allocation 
should  be  based  on  sensory,  cognitive,  and  behavioral  capabilities  of  the 
pilot  population.  When  the  aid  determines  that  task  demands  exceed  pilot 
capabilities,  that  task  should  be  off-loaded  to  the  aid.  Demands  on  the 
pilot’s  resources,  however,  must  be  considered  in  the  context  of  multitask 
performance. 
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Empirical  Support:  The  LOA  methodology  has  not  been  validated. 

4.4.1  .f  Three  primary  levels  of  adaptation  are:  transformation  (changing  a  task  to 
make  it  easier); partitioning  (sharing  a  task  betv/een  aid  and  operator); 
and  allocation  (distributing  tasks  between  operator  and  aid)  (Rouse  and 
Rouse,  1983). 

Explanation:  These  three  methods  of  adaptive  aiding  were  Introduced  at 
the  beginning  of  this  review  and  provide  a  useful  method  of  classifying 
types  of  adaptive  aiding. 

Empirical  Support: 

4.4.1.  g  An  aiding  system  must  be  "adaptive"  (i.e.,  not  mission-mode  or  function 

automation  based)  to  be  able  to  compensate  for  operator  errors.  Flexible 
automation  is  necessary  to  meet  this  requirement  (Andes,  1987). 

Explanation:  To  compensate  for  operator  error,  aiding  systems  must  be 
flexible,  must  have  the  capacity  to  monitor  operator  error,  must  possess 
knowledge  of  error  compensation  procedures,  and  must  have  the  ability  to 
influence  the  system  at  the  necessary  level  of  input. 

Empirical  Support:  Although  this  guideline  has  been  conceptually 
developed,  it  has  not  been  validated  in  an  aiding  setting. 

4.4.1.  h  Adaptive  automation  may  benefit  pilot  performance  by  preventing 

performance  decrements  related  to  long  term  monitoring,  loss  of  situation 
awareness  and  manual  skill  degradation  (Morrison  et  al.  1990). 

Explanation:  An  adaptive  approach  may  decrease  the  effects  of  vigilance 
and  thereby  minimize  performance  decrements:  the  pilot  remains  in  active 
control  of  the  system  instead  of  becoming  a  passive  observer  of  the 
system. 

Empirical  Support: 
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4.4.1. 1  Use  decision  aiding  to  reduce  threat  categones  in  classification  tasks, 

reduce  response  categories  in  translation  tasks,  and  reduce  the  number  of 
available  controls  in  selection  tasks,  thereby  making  the  tasks  easier  to 
perform  (Barnes,  1985). 

Explanation:  Adaptive  decision  aiding  can  transform  tasks  and  make  them 
easier  to  perform.  This  is  yet  another  method  of  maximizing  performance 
through  the  use  of  adaptive  aiding  systems. 

Empirical  Support:  This  method  of  decision  aiding  has  to  be  empirically 
validated. 

4.4.2  Implications  on  Workload 

The  methods  used  to  implement  aiding  in  a  system  may  either  increase  or 
decrease  operator  workload  levels.  The  operator,  for  example,  may  experience 
increases  in  workload  if  he  is  required  to  obtain  aiding  information  from  a  display  that 
is  cluttered  with  irrelevant  Information.  It  is  therefore  Important  to  determine  what 
methods  of  adaptation  optimize  operator  workload  associated  with  different  methods 
of  adaptation. 

Guideline 

4.4.2.a  In  multiple  task  situations,  attempt  to  distribute  tasks  among  different 

resources  of  the  human  to  achieve  higher  overall  performance  and  reduce 
workload  on  the  human  (Derrick,  1 988). 

Explanation:  In  an  approach  that  is  analogous  to  the  partitioning  method 
of  adaptation.  Derrick  proposes  that  tasks  should  be  distributed  among 
different  human  processing  resources.  The  rationale  is  that  distributing 
tasks  among  processing  resources  will  prevent  the  overload  of  a  single 
resource,  thus  optimizing  workload  and  performance. 

Empirical  Support:  This  guideline  is  based  on  Wickens’  (1984)  Multiple 
Resource  Theory. 
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4.4.3  Implications  on  Operator  Acceptance 

The  methods  used  to  implement  aiding  in  a  system  have  a  direct  affect  on 
operator  acceptance  because  these  methods  determine  how  the  operator  and  aid 
will  interact. 

Guideline 

4.4.3.a  The  beliefs  and  philosophies  of  designers  are  likely  to  affect  the  types  of 
aiding  chosen.  Consider  whether  the  resulting  aiding  system  will  be  either 
human-requirements  centered  or  mission-requirements  centered  and 
design  accordingly  (Andes  and  Rouse,  1991). 

Explanation:  The  designers'  approach  to  aiding  is  likely  to  affect  operator 
acceptance  of  aiding  systems  because  the  approach  will  be  visible  in  the 
final  design.  Andes  and  Rouse  suggest  that  designers  approach  design 
by  determining  whether  a  system  will  be  human-requirements  centered  or 
mission-requirements  centered.  This  approach  may  foster  operator 
acceptance  because  design  is  based  on  the  orientation  of  the  user. 

Empirical  Support:  This  is  the  first  formulation  of  such  an  approach  to 
aiding  design.  Further  research  must  be  conducted  to  determine  how 
design  philosophy  affects  system  design,  as  well  as  how  beliefs  might  be 
used  to  specify  system  behavior  based  on  user  or  mission  requirements. 

4.4.4  Implications  on  Situation  Assessment 

We  were  not  able  to  extract  any  guidelines  which  addressed  methods  of 
adaptation  in  the  context  of  situation  assessment  from  the  references  included  in  the 
bibliography. 

4.4:  Section  Summary 

This  section  discussed  methods  that  designers  may  use  to  specify  aiding 
requirements  in  the  design  of  aiding  systems.  These  guidelines  propose  that 
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systems  engineering  methods,  critical-event  logic,  levels  of  autonomy,  levels  of 
adaptation,  and  decision  aiding  techniques  may  be  used  by  designers  to  determine 
their  approach  aiding  system  design.  These  guidelines  suggest  that  designers 
should:  a)  distribute  tasks  among  the  operators’  information  processing  resources, 
and  b)  design  the  aiding  system  to  satisfy  either  operator  requirements  or  mission 
requirements. 


4.5  How  is  Adaptation  Done? 

There  are  two  ways  to  determine  the  needs  for  adaptation  in  a  given  situation. 
One  method  involves  measurement  of  operator  or  system  performance,  and  the 
other  involves  modeling  operator  resources,  intentions,  or  system.  These  methods 
provide  designers  with  the  information  necessary  to  determine  how  to  best 
implement  adaptation  within  a  system. 

4.5.1  implications  on  Performance 

The  ways  in  which  designers  approach  aiding  the  operator  may  affect  system 
performance.  Subsequently,  the  final  implementation  of  the  aid  may  facilitate  or 
inhibit  required  levels  of  operator  performance.  It  is  therefore  important  to  consider 
which  methods  of  assessing  aiding  needs  are  accurate. 

Guidelines 

4.5.1. a  Designers  are  interested  in  relationships  among  tasks  and  appropriate 

types  of  aiding  appropriate  within  a  specified  task  context  (Andes  and 
Rouse,  1991). 

Explanation:  Experimental  and  subject  debriefing  data  in  the  Andes  and 
Rouse  investigation  showed  that  designers  need  information  about  how 
the  type  of  aiding  implemented  in  a  system  may  affect  task  performance. 
Designers  also  consider  contextual  influences  on  how  the  type  of  aiding 
will  affect  human  and  system  performance. 
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Empirical  Support:  The  study  by  Andes  and  Rouse  supports  this  and  the 
following  two  guidelines. 

4.5.1.  b  Designers  are  interested  in  information  about  appropriate  invocation 

criteria  for  different  types  of  aiding  (Andes  and  Rouse,  1991). 

Explanation:  The  study  revealed  that  designers  find  it  important  to 
consider  how  and  when  aiding  should  be  invoked,  and  how  different  types 
of  aiding  might  affect  system  performance.  For  example.  If  aiding  is  not 
automatic  and  operators  find  it  difficult  and  time-consuming  to  invoke  the 
aid,  system  performance  is  likely  to  suffer. 

Empirical  Support:  The  study  supports  this  guideline. 

4.5.1.  c  Designers  are  likely  to  value  data  that  compare  types  of  aiding  and 

appropriate  invocation  criteria  as  a  function  of  types  of  task  and  the 
motivation  for  aiding.  Designers  also  prefer  to  make  decisions  based  on 
specific  and  concrete  information  rather  than  general  principles  (Andes 
and  Rouse,  1991). 

Explanation:  This  guideline  ties  the  previous  two  together.  As  stated, 
designers  believe  that  system  performance  is  likely  to  be  affected  by  the 
type  of  aiding  implemented  and  the  way  it  is  invoked.  Thus,  designers  find 
it  important  to  consider  these  factors  in  the  context  of  the  task  and  the 
conditions  for  invoking  the  aid.  Designers  probably  prefer  basing  design 
decisions  on  specific  and  concrete  information  because  they  can  be  more 
confident  that  they  will  make  correct  decisions  with  this  type  of  information. 

Empirical  Support:  The  study  supports  this  guideline. 

4.5.1.  d  The  importance  of  aiding  any  task  changes  depends  on  the  current 

demands  of  the  situation.  Therefore,  the  level  of  aiding  and  the  ways  the 
human  and  aid  interact  should  change  with  changes  in  task  demands 
(Rouse,  1988). 
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Explanation:  In  dynamic  situations,  the  criticality  of  a  given  task  changes 
with  regard  to  the  demands  of  the  current  situation.  This  implies  that  the 
level  of  aiding  and  human-aid  interaction  must  also  change  with  regard  to 
situational  demands  In  order  to  optimize  system  performance. 

Empirical  Support:  This  guideline  is  the  basic  premise  of  the  concept  of 
adaptive  aiding. 

4.5.1. e  The  operator's  aiding  needs  depend  on  both  impending  and  recently 
completed  task  demands  (Rouse,  1988). 

Explanation:  It  has  been  found  that  the  demands  of  both  anticipated  and 
completed  tasks  affect  the  operator’s  needs  for  aiding  (Morris  and  Rouse, 

.  1 986).  If  a  difficult  task  has  just  been  completed,  for  example,  an 
operator’s  ability  to  perform  a  current  task  may  be  decreased.  The 
existence  of  these  "carryover"  effects  implies  that  aiding  should  be  adapted 
to  both  passing  and  forthcoming  task  demands. 

Empirical  Support:  Support  is  provided  in  Morris  and  Rouse  (1986). 

4.5.1  .f  The  degree  of  task  structure  will  dictate  the  accuracy  with  which  Inferences 
of  activities,  awareness,  and  intentions  can  be  made  when  using  models 
as  a  basis  of  adaptation  (Rouse,  1988). 

Explanation:  Tasks  with  substantial  levels  of  variability  in  user 
performance  are  not  suitable  for  model-based  adaptation.  For  example,  if 
a  model  cannot  account  for  the  way  a  user  customizes  aiding,  the  model 
cannot  accurately  predict  when  during  task  performance  adaptation  should 
be  invoked. 

Empirical  Support:  This  is  a  conceptual  guideline;  it  does  not  yet  have 
empirical  support. 

4.5.1  .g  Secondary  indices  of  user  and  system  behavior  may  be  used  as  proxy 
measures  of  the  primary  indices  of  concern  (Rouse,  1988). 
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Explanation:  This  guideline  implies  that  indirect  measures  can  be  used  to 
obtain  information  about  user  and  system  behavior,  which  may 
consequently  indicate  the  need  for  aiding.  For  example,  workload  may  not 
be  measured  directly  because  it  is  difficult  to  quantify  with  one  objective 
measure.  This  guideline  would  suggest  using  temporal  patterns  of  user 
and  system  behavior  to  indicate  when  workload  is  high  and  therefore, 
when  aiding  is  necessary. 

Empirical  Support: 

4.5.1. h  Input  information  about  operator  behavior  (e.g.,  current  resource  loading, 

manual  performance,  operator  errors,  and  knowledge  of  intent)  can  be 
used  as  aid-initiated  aiding  thresholds  (Andes,  1987). 


Explanation:  There  are  several  ways  to  indicate  to  a  system  when  aiding 
should  be  applied.  Andes  suggests  using  specific  levels  of  resource 
loading,  performance,  error  frequency  and  deviations  from  intent  as 
thresholds  for  aiding  systems  to  initiate  aiding.  This  method  is  not  likely  to 
degrade  system  performance  because,  by  gathering  operator  performance 
information  via  secondary  measures,  the  system  remains  unobtrusive  to 
operator  performance. 


Empirical  Support: 

4,5.1. i  When  assessing  instantaneous  tracking  performance  (generally)  RMS 

tracking  error  is  nd  a  good  estimator. 

Explanation:  Guideline  4.5.1. i  appears  valid  in  most  control  situations,  but 
Morris  and  Rouse  have  shown  that  RMS  tracking  error  analysis  is  not  a 
good  estimator  of  tracking  performance.  They  show  that  context  has  a 
strong  influence  on  tracking  performance  and  that  RMS  error  analyses  do 
not  account  for  this  influence.  Instead,  they  suggest  analysis  of  event 
arrival  rates  as  a  secondary  index  of  possible  speed-accuracy  tradeoffs  in 
performance. 

Empirical  Support:  Support  is  provided  in  the  study. 
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4.5.1. j  Adaptive  aiding  interfaces  should  allow  the  user  to  receive  direct 

assistance  in  planning  how  to  carry  out  the  intended  task.  This  can  be 
achieved  by  modeling  system  tasks  and  representing  this  hierarchy  within 
the  aiding  system  (Tyler  and  Treu,  1989). 

Explanation;  Modeling  system  tasks  and  representing  a  task  hierarchy 
within  the  aiding  system  interface  may  be  a  good  way  to  achieve  adaptive 
aiding.  Users  have  direct  access  to  aiding  information  when  it  is  displayed 
in  the  interface  in  a  structured  form.  In  addition,  management  models  of 
system  communication  and  collaboration  can  be  useful  in  design,  and 
have  been  commonly  specified  by  different  researchers. 

Empirical  Support:  This  guideline  is  supported  in  the  study. 

4.5.1  .k  A  matrix  of  models  approach  can  be  used  to  estimate  point  performance  of 

the  operator  (e.g.,  signal  detection  probability,  choice  selection  reaction 
time,  choice  selection  speed/accuracy  tradeoff,  reach  and  touch  timing 
models,  etc.).  Prediction  of  performance  requires  a  more  integrated 
approach,  however.  (Andes,  1987). 

Explanation:  By  creating  matrices  of  models,  Andes  suggests  that  it  would 
be  possible  to  incorporate  such  predictors  of  performance  as  signal 
detection  probability,  choice  selection  reaction  time,  speed/accuracy  trade¬ 
off  assessment,  and  reach-and-touch  timing  models  into  estimations  of 
performance.  Such  a  matrix  would  be  useful  for  aid-initiated  adaptation  of 
system  performance  at  particular  points  in  time.  The  matrix  approach 
would  not  be  useful  if  predictions  of  performance  were  required. 

Empirical  Support:  Validation  in  realistic  task  environments  has  not  been 
performed. 

4.5.1. 1  Use  pilot  performance  models  to  understand  pilot  behavior  and  trigger 
aiding  (Parasuraman  et  al.  1990).  These  models  should  be  wide-scoped 
and  non-intrusive.  Examples  are:  Intent  inferencing  (Rouse,  et  al,  1987); 
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Multiple  Resource  Theory  (Wickens,  1984);  queueing  models  (Rouse  and 
Chu,  1979):  and  central  executive  models  (Barnes,  1985). 

Explanation:  Parasuraman  et  al.  suggest  that  designers  use  pilot 
performance  models  to  explain  pilot  behavior.  Additionally,  they  suggest 
implementing  such  models  in  aiding  systems  as  a  way  to  notify  the  aid  to 
initiate  aiding.  Performance  models  supply  more  information  than 
behavioral  measurement  methods  because  they  account  for  a  larger 
number  of  workload  parameters  and  may  be  adjusted  for  Individual 
operators.  Examples  of  performance  models  are  listed  in  the  guideline. 

Empirical  Support: 

4.5.1.  m  Modeling  of  human  behavior  for  aid-initiated  intervention  should  at  least 

include:  task  execution  goal  states;  environment  representation 
(graphical);  situation  assessment  information;  and  planning  and 
commitment  logic  (Andes  and  Hunt,  1989). 

Explanation:  Andes  and  Hunt  propose  several  information  requirements 
for  modeling  human  behavior  in  aid-initiated  Intervention  situations.  The 
authors  propose  that  in  order  to  effectively  intervene,  an  aiding  system 
would  have  to  possess  information  about  the  operator’s  goals,  the  problem 
context,  the  current  situation,  and  methods  for  behavioral  prediction. 

These  types  of  information  were  used  in  a  limited  experimental  control 
system  to  initiate  aid  interaction,  but  the  model  of  human  behavior 
developed  was  crude. 

Empirical  Support: 

4.5. 1. n  Implementation  of  models  of  human  performance  in  an  aiding  system  can 

increase  human  performance  without  degrading  performance  of  the  aid 
(Revesman  and  Greenstein,  1986). 

Explanation:  Revesman  and  Greenstein  have  Implemented  models  of 
human  performance  in  an  aiding  system  and  have  studied  how  the  use  of 
such  models  affects  system  performance.  This  guideline  was  the  result  of 
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an  investigation  that  analyzed  the  use  of  model-based  communication  to 
facilitate  dynamic  task  ailocation  in  a  multi-task  environment.  They  found 
that  the  use  of  communication  models  supplemented  human  performance 
and  did  not  affect  performance  of  the  aid. 

Empirical  Support;  This  guideline  was  validated  in  the  Revesman  and 
Greenstein  study. 

4.5.2  Implications  on  Workload 

Optimization  of  workload  is  necessary  to  successful  aiding  system 
implementation.  The  designers'  approach  to  adaptation  has  a  direct  bearing  on 
operator  workload  levels. 

Guidelines 

4.5.2. a  The  aid  should  provide  aid-initiated  intervention  so  as  not  to  increase  the 

already  high  pilot  workload  (Krobusek,  Boys,  and  Palko,  1989). 

Explanation:  The  basic  premise  of  adaptive  aiding  is  to  aid  operators’ 
performance  of  tasks,  to  do  so  without  requiring  operators  to  ask  for 
assistance,  and  for  an  aid  to  adjust  to  the  individual  operator  and  the 
context  of  the  situation.  This  goal  of  minimal  pilot  workload  would  not  be 
achieved  if  the  pilot  was  required  to  invoke  aiding  (Morris,  Rouse  and 
Ward,  1985;  Rouse,  1988). 

Empirical  Support: 

4.5.2. b  Compatibility  between  stimulus  (S)  and  response  (R)  can  be  achieved  by: 

close  geometric  proximity  of  S  and  R  equipment,  isomorphic  relationships 
between  S/R  configurations,  and  cognitive  similarity  of  stimulus  code  and 
response  modality  (Barnes,  1981). 

Explanation:  Pilot  workload  in  pilot-computer  interactive  situations  may  be 
minimized  through  design  of  the  interface.  In  particular,  the  way  the 
stimulus  (S)  is  perceived  and  the  response  (R)  it  requires  should  be 
compatible.  Barnes  provides  several  ways  to  achieve  S/R  compatibility  In 
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the  interface.  Among  these  methods  are  close  proximity  of  S  and  R 
equipment,  isomorphic  relationships  between  S/R  configurations,  and 
cognitive  similarity  of  S  and  R  modalities. 

Empirical  Support:  It  is  believed  that  these  methods  would  facilitate  pilot- 
computer  interaction  by  minimizing  workload,  but  this  guideline  has  not 
been  widely  validated. 

4.5.2. C  Dynamic  assessment  of  pilot  workload  provides  the  rationale  for 

implementing  automation  in  an  adaptive  rather  than  fixed  manner.  Direct 
assessment  (e.g.,  psychophysical  measures)  and  indirect  assessment 
(e.g.,  subjective  measurement)  are  alternatives  (Parasuraman  et  al.  1990). 

Explanation:  Methods  for  dynamic  assessment  of  pilot  workload  have 
been  discussed  in  detail  by  Parasuraman  et  al.  and  are  presented  In 
Guideline  4.5.2.e.  The  authors  suggest  that  dynamic  workload 
assessment  is  necessary  In  adaptive  aiding  systems  because  it  would 
allow  automation  to  adapt  to  the  changing  needs  of  the  pilot  in  a  given 
situation.  Further,  dynamic  aiding  Is  likely  to  lead  to  better  system 
performance  than  fixed  or  static  aiding. 

Empirical  Support:  Although  subjective  measurements  of  workload  have 
undergone  recent  Improvements,  the  use  of  these  measures  as  the  basis 
of  aid-initiated  adaptation  must  be  further  researched.  Derrick  (1988)  and 
Wickens  (1 984)  have  also  addressed  subjective  and  dynamic  measures  of 
workload. 

4.5.2. d  Dynamic  workload  can  be  assessed  as  a  function  of:  current  subjective 

distance  from  a  desired  goal,  time  to  reach  the  goal,  and  level  of  operator 
effort  required  to  achieve  the  goal  given  the  time  available  (Parasuraman 
et  al.  1990;  Hancock  and  Chignell,  1988). 

Explanation:  Parasuraman  et  al.  have  found  that  dynamic  workload 
assessment  is  affected  by  the  several  factors  listed  in  this  guideline.  The 
perceived  distance  from  a  desired  goal,  time  required  to  reach  the  goal, 
and  required  level  of  operator  effort  all  compound  the  operator's  level  of 
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workload.  Increases  in  any  of  these  factors  correspond  to  increases  in 
workload. 

Empirical  Support:  Hancock  and  Chignell  (1988)  have  provided  the 
necessary  validation  of  these  measures  in  dynamically  determining  levels 
of  workload  in  particular  contexts. 

4.5.2. e  Dynamic  psychophysiological  assessment  can  be  used  as  a  workload 

measure.  Methods  of  measurement  include:  pupillary  dilation,  heart  rate, 
EEG,  ERP,  and  eye  scanning  and  fixation  measurements  (Parasuraman  et 
al.  1990). 

Explanation:  Although  these  psychophysiological  methods  of  dynamic 
workload  assessment  can  be  obtained  continuously  (a  truly  dynamic 
assessment),  they  are  expensive,  potentially  Intrusive,  and  of  unknown 
reliability.  Using  them  in  conjunction  with  behavioral  measures,  however, 
may  provide  additional  information  than  other  workload  assessment 
methods  alone. 

Empirical  Support:  The  reliability  of  these  methods  requires  further 
investigation.  Further  research  is  also  required  to  determine  how  well 
psychophysiological  methods  correlate  with  static  and  subjective  workload 
assessment  methods. 

Implications  on  Operator  Acceptance 

The  methods  of  aiding  implementation  affect  the  operator  at  the  interface  to 
the  system.  It  is  necessary  for  the  operator  to  be  able  to  communicate  with  the 
system  at  a  desirable  level  of  interaction  in  order  to  achieve  operator  acceptance. 

Guidelines 

4.5.3. a  The  aid  must  assist  the  operator  without  interfering  In  those  tasks  that  the 

operator  is  executing,  and  do  so  efficiently  (Andes.  1987). 
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Explanation:  Simulation  observations  suggest  that  operators  will 
deactivate  an  aid  if  they  perceive  that  it  is  interfering  with  performance  of 
their  tasks.  This  will  occur  even  if  the  aid  is  assisting  performance  of 
important  system  tasks.  Designers  must  be  aware  of  this  issue  during  the 
development  of  their  aid  implementation  plans  in  order  to  promote 
operator  acceptance  of  the  aid. 

Empirical  Support:  Although  the  validity  of  this  guideline  may  seem 
obvious,  it  has  not  been  tested  empirically. 

4.5.3. b  The  system  must  support  allocation  from  human  to  machine:  It  must  be 

responsive  to  pilot  workload,  pilot  preference,  current  pilot  activity,  and 
mission  contingencies  for  reallocation  (Krobusek  et  al.  1989). 

Explanation:  Intelligent  operator  interface  systems,  including  the  Pilot's 
Associate  system,  have  been  developed  on  the  premise  that  an  aid  should 
support  the  allocation  of  tasks  from  the  human  to  the  aid.  Krobusek  et  al. 
propose  that  in  order  to  fulfill  this  premise,  an  aiding  system  must  adapt  to 
changing  levels  of  workload,  pilot  preferences,  current  pilot  behavior,  and 
previously  defined  contingencies  for  task  reallocation.  Operator 
acceptance  may  be  facilitated  through  adherence  to  this  guideline. 

Empirical  Support:  Additional  research  is  needed  to  determine  the  extent 
to  which  operator  acceptance  would  be  promoted  by  the  designer's 
consideration  of  the  above  listed  factors. 

4.5.3. C  Adaptive  aiding  systems  should  allow  the  user  to  execute  an  action  by 

directly  expressing  high  level  intentions.  These  intentions  should  be 
recognizable  by  the  aiding  system  as  explicit  plans  of  action  executable 
within  the  system  (Tyler  and  Treu,  1989)  (Rouse,  Geddes  and  Curry, 

1987). 

Explanation:  The  execution  of  actions  by  direct  expression  of  high-level 
intentions  was  also  proposed  by  Rouse,  Geddes  and  Curry  (1987)  and 
Noah  and  Halpin  (1986).  If  direct  expression  of  intent  is  to  be  possible  In 
an  aiding  system,  the  system  has  to  be  capable  of  recognizing  intentions 
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as  explicit  pians  for  action.  Therefore,  designers  must  represent  operator 
tasks  as  plans  to  the  aiding  system.  This  also  requires  that  the  system  has 
enough  domain  knowledge  to  account  for  the  relationship  between  system 
components  and  system  capabilities.  It  seems  possible  that  operator 
acceptance  of  an  aiding  system  will  be  fostered  if  the  system  is  able  to 
correctly  interpret  intentions  as  plans  of  action. 

Empirical  Support:  Expression  of  operator  Intent  within  a  system  is  an 
ambitious  goal  and  has  yet  to  be  successfully  validated.  Geddes  (1 989), 
however,  has  evaluated  expression  of  implicit  operator  intent  within  an 
experimental  control  task. 

4.5.3. d  The  aid  must  reflect  the  operator's  mental  model  of  an  ideal  executive 

assistant  for  optimal  system  performance.  The  aid  can  determine  this 
estimate  by  user  performance  modeling  or  current  system  status 
measurement  (Andes,  1987). 

Explanation:  Operator  acceptance  may  be  promoted  if  an  operator  feels 
as  if  the  aiding  system  approximates  his  image  of  an  ideal  assistant.  This 
requires  that  the  system  knows  what  the  operator  expects  of  system 
performance.  If  an  aid  cannot  incorporate  at  least  a  rough  estimate  of  the 
operator's  mental  model  of  an  ideal  assistant,  then  it  is  likely  to  fail  the  test 
of  user  acceptance.  Methods  for  determining  models  of  the  ideal  executive 
assistant  include  the  use  of  performance  modeling  or  current  system  status 
measurement. 

Empirical  Support:  This  guideline  has  been  developed  from  anecdotal 
evidence  In  the  Pilot's  Associate  evaluation,  and  therefore  requires  a 
substantial  amount  of  additional  research. 

4.5.3. e  Use  adaptive  pattern  classification  techniques  to  track  a  decision  maker’s 

responses  in  decision  support  systems.  This  information  can  be  used  to 
learn  the  decision  maker’s  utilities  and  provide  customized  decision 
support  based  on  dynamic  utilities  for  alternatives  (Weisbrod  et  al.  1977). 
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Explanation:  Weisbrod  et  al.  created  models  of  decision  making  using  the 
adaptive  pattern  classification  method.  These  models  were  then  used  to 
estimate  decision  maker  utilities,  and  to  provide  customized  decision 
support.  An  experiment  showed  that  subjects  who  received  aiding  based 
on  utility  estimation  made  significantly  fewer  deviations  from  their  own 
maximum  expected  utility,  and  were  significantly  less  variable  In  their 
responses  than  unaided  subjects.  Greater  dedsion  output  was  also 
observed  in  the  aided  subjects. 

Empirical  Support:  Although  positive  performance  measures  are  likely  to 
lead  to  operator  acceptance  of  an  aid,  additional  validation  should  be 
performed  to  insure  that  this  type  of  aiding  is  indeed  operator  accepted. 
Madni  (1988)  discusses  decision  maker  utility  estimation  techniques  in 
more  detail. 

4.5.4  implications  on  Situation  Assessment 

Aiding  implementation  methods  affect  the  ability  of  the  operator  to  assess 
situations  because  these  methods  determine  how  the  operator  will  interact  with  the 
system.  The  operator  must  be  able  to  make  efficient  situation  decisions  based  on  the 
information  provided  by  the  aid,  and  further,  the  aid  must  not  overload  the  operator 
with  demands  on  his  attention. 

Guideline 

4.5.4.a  The  aid  must  have  a  framework  by  which  it  limits  its  demands  on  pilot 

attention  to  critical  interactions,  allowing  the  pilot  to  maintain  awareness  of 
system  activities  (Krobusek  et  al.  1989). 

Explanation:  It  is  imperative  that  pilots  remain  aware  of  system  activities  as 
well  as  events  occurring  outside  the  system.  For  an  adaptive  aid  to 
successfully  maintain  pilot  awareness,  the  aid  must  limit  the  demands  that 
it  places  on  pilot  attention.  Such  demands  should  be  limited  to  predefined, 
critical  system  interactions.  The  LOA  (Levels  of  Autonomy)  paradigm 
proposed  by  Krobusek  et  al.  does  indeed  limit  demands  placed  on  pilot 
attention,  allowing  pilots  to  specify  before  the  start  of  a  mission  when  they 
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would  prefer  to  be  notified  of  critical  situations,  and  when  they  would  prefer 
that  the  aid  initiate  aiding  automatically.  Andes  (1990)  and  Lind  (1989) 
address  aiding  implementation  issues  that  may  affect  pilot  ability  to  assess 
situations.  They  are  discussed  in  Section  4.2.3. 

Empirical  Support:  Further  validation  of  using  LOA  to  insure  pilot  situation 
awareness  during  aiding  Is  required. 

4.5:  Section  Summary 

This  section  discussed  how  designers  may  implement  the  Invocation  of  aid  In 
aiding  systems.  These  guidelines  promote  the  use  of  operator  performance  criteria, 
the  influence  of  both  impending  and  recently  completed  tasks,  operator 
plahning/Intention  and  models  of  human  performance  to  guide  design  decisions 
about  aid  invocation.  In  addition,  research  suggests  that  aid-initiated  intervention  is 
critical  in  aiding  systems.  These  guidelines  suggest  that  designers  should:  a) 
minimize  the  demands  of  aiding  systems  on  operator  attention,  and  b)  implement 
systems  that  are  both  perceptive  and  responsive  to  operator  intention. 


4.6  What  Is  the  Nature  of  Aid-Operator  Communication? 

The  operator  and  the  aid  communicate  Information  either  before  or  in 
response  to  the  effects  of  adaptation.  This  communication  may  be  either  explicit, 
comprising  part  of  the  operator-aid  interaction  task,  or  it  may  be  implicit,  occurring 
"behind  the  scenes"  during  task  performance.  Communication  between  the  aid  and 
the  operator  is  a  paramount  issue.  Efficient  communication  can  significantly 
increase  performance  on  several  dimensions  (i.e.,  situation  assessment, 
coordination  of  activities).  Inefficient  communication  can  result  in  degraded  system 
performance  (e.g.,  repeated  activities,  confusion  about  task  responsibility)  and 
ultimately,  system  breakdown. 

4.6.1  Implications  on  Performance 

The  communication  between  aid  and  operator  may  affect  system 
performance.  If  the  aid  or  the  operator  do  not  receive  accurate  or  complete 
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information  during  communication,  decrements  in  performance  will  occur.  It  is 
necessary  therefore  to  insure  that  optimal  methods  of  communication  are 
implemented  within  adaptive  aiding  systems. 

Guidelines 

4,6.1  .a  The  cost  of  explicit  communication  (e.g.,  workload  and  time  required)  with 
the  user  should  be  compared  with  the  cost  of  adaptation  errors  (i.e.,  misses 
and  false  alarms).  This  guideline  is  applicable  primarily  to  user  error 
events  (Rouse,  1988). 

Explanation;  The  process  of  communication  between  the  aid  and  the 
operator  will  affect  system  performance.  Revesman  and  Greenstein  (1986) 
addressed  the  costs  of  the  different  types  of  communication  with  regard  to 
user  error  events.  Rouse  has  further  addressed  how  communication  may 
affect  user  and  system  performance.  Apparently,  although  explicit 
communication  may  Improve  user  and  system  performance  in  terms  of 
reduced  misses  and  false  alarms,  it  Is  expensive  in  terms  of  added 
workload  and  time  required  for  communication.  Implicit  communication, 
though  less  expensive,  may  not  improve  performance  to  the  same  extent' 
as  explicit  communication.  Rouse  suggests  that  designers  should 
compare  the  cost  of  explicit  communication  with  the  user  to  the  cost  of 
aiding  errors;  the  system  should  use  the  cheapest  communication  possible 
resulting  in  a  tolerable  level  of  aiding  performance. 

Empirical  Support: 

4.6.1. b  The  aid  must  justify  advice  to  human  based  on  fallible  algorithms.  Whether 
a  human  decides  to  use  advice  depends  on  how  he  chooses  to  use  the 
aid,  and  the  discretion  he  uses  In  accepting  or  rejecting  the  advice  (Lehner 
et  al.  1989). 

Explanation;  Aiding  systems  base  their  advice  on  algorithms  that  are  not 
perfectly  accurate.  Operators  may  not  be  aware  of  this  fact,  but  may 
nevertheless  be  in  a  position  where  they  must  decide  whether  or  not  to 
accept  the  aid's  advice.  In  an  empirical  setting,  Lehner  et  al.  found  that 
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task  performance  is  best  wherf  the  aid's  advice  is  routinely  accepted  or 
rejected,  when  operators  regularly  follow  or  Ignore  aiding  advice.  These 
results  imply  that  operators  who  put  only  some  trust  in  the  aid's  advice  will 
not  perform  as  well  as  operators  who  put  all  of  their  trust  or  none  of  their 
trust  in  the  aid.  For  optimal  performance,  operators  should  be  aware  of  this 
phenomenon. 

Empirical  Support:  The  Lehner  et  al.  study  supports  this  guideline  by 
showing  that  performance  is  affected  by  operators'  discretion  in  using  an 
aid's  advice.  Morris  and  Rouse  (1986)  discuss  user  discretion  in  the 
context  of  information  requirements  for  communication. 

4.6.1.  c  Variable  display  formats  are  useful  in  situations  where  operators  have 

habituated  to  certain  types  of  signals  and  response  latencies  are 
increasing  (Forester,  1987). 

Explanation:  This  guideline  implies  that  communication  between  operator 
and  aid  may  be  facilitated  by  changes  in  display  format  once  the  operator 
begins  to  habituate  to  one  type  of  format.  Forester  developed  this  concept 
with  regard  to  Wickens'  (1984)  Multiple  Resource  Theory.  Although  it  may 
facilitate  communication,  changing  the  way  an  operator  interacts  with  an 
interface  by  changing  the  way  information  is  presented  may  or  may  not 
have  detrimental  effects  on  system  or  human  performance. 

Empirical  Support: 

4.6.1. d  The  dialogue  between  the  adaptive  system  and  the  user  must  be 

appropriate  to  the  user  and  the  application  for  which  the  system  is 
designed.  Consider  target  user  vocabulary  and  capabilities  of  each  user 
population  to  facilitate  maximum  Information  bandwidth  (Norcio  and 
Stanley,  1989). 

Explanation:  Norcio  and  Stanley  have  addressed  the  methods  by  which 
communication  may  occur  in  an  adaptive  system.  Through  experiments 
they  have  found  that  the  dialogue  between  the  system  and  the  user  must 
be  appropriate  to  the  user  and  to  the  tasks  for  which  the  system  was 
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designed.  To  insure  optimal  information  throughput  during  aid-user 
communication,  designers  should  consider  the  target  user  vocabulary  and 
the  capabilities  of  different  user  populations,  and  should  design  the  aiding 
system  accordingly.  This  guideline  is  especially  useful  for  systems  used 
by  several  user  populations. 

Empirical  Support:  The  Norcio  and  Stanley  study  provides  support  for  this 
guideline.  In  addition,  Morris  and  Rouse  (1986)  have  addressed  methods 
of  communication  and  their  effects  on  performance  with  regard  to  dynamic 
task  allocation. 

4.6. 1. e  Model-based  communication  (implicit)  between  operator  and  aid  Is 

desired  when  operator  workload  is  high,  or  when  minimization  of  a 
particular  measure  of  system  cost  (i.e.,  response  time)  is  necessary 
(Revesman  and  Greenstein,  1986). 

Explanation:  It  is  possible  to  use  models  of  operator-aid  communication  to 
Initiate  aiding  when  operators  are  under  high  workload  conditions.  This 
will  benefit  system  performance  because  model-based  communication  is 
implicit  to  the  system  and  does  not  require  operator  attention.  Further, 
implicit  communication  methods  impose  minimal  system  performance 
costs,  and  may  provide  the  basis  for  aiding  intervention  thresholds,  intent 
inferencing,  and  tailored  resource  modeling.  Though  little  empirical 
support  for  this  guideline  exists,  the  implicit  communication  that  is  afforded 
by  modeling  has  been  espoused  by  Rouse  (1988)  and  Rouse,  Geddes 
and  Curry  (1987). 

Empirical  Support: 

4.6. 1. f  Use  model-based  communication  between  the  operator  and  aid  when  the 

aid  cannot  communicate  its  actions  to  the  operator.  This  enables  the  aid  to 
work  around  the  operator  and  reduce  the  number  of  redundant  actions 
(Revesman  and  Greenstein,  1986). 

Explanation:  Revesman  and  Greenstein  suggest  using  model-based 
communication  when  explicit  communication  between  operator  and  aid  is 
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not  possible.  Model-based  communication  allows  the  aid  to  determine 
possible  operator  actions.  If  the  model  predicts  that  the  operator  will 
perform  certain  tasks,  the  aid  will  perform  other  necessary  tasks.  Not  only 
will  this  method  maximize  system  efficiency,  it  will  reduce  the  number  of 
redundant  actions  that  the  aid  might  perform. 

Empirical  Support:  Validation  of  this  guideline  is  necessary  to  find  whether 
operator  ignorance  of  the  aid’s  functioning  will  negatively  affect  system 
performance. 

4.6.1.  g  Where  possible,  an  analysis  should  be  conducted  to  determine  the  extent 

of  loss  of  system  performance  due  to  Imperfect  model  validity  (for 
communication)  traded  against  the  loss  of  system  performance  due  to  time 
required  for  explicit  dialogue  (Revesman  and  Greenstein,  1986). 

Explanation:  Revesman  and  Greenstein  suggest  analyzing  the 
effectiveness  of  model-based  communication  when  time  constraints 
permit.  This  will  allow  the  system  to  determine  how  performance  is 
affected  by  improper  fit  of  the  model  to  the  task  context.  The  system  should 
then  determine  how  performance  is  affected  by  the  time-related  cost  of 
explicit  operator-aid  dialogue.  The  system  should  note  whether  the  cost  to 
performance  of  imperfect  model  fit  or  explicit  communication  Is  lower,  and 
should  implement  that  alternative. 

Empirical  Support: 

4.6.1.  h  A  two  stage  model  can  be  used  to  predict  decision  making  and  action  in 

human-computer  interaction:  1.  Human  event  detection  is  modeled  by 
generating  the  probability  of  response  to  an  event;  2.  Control  actions  are 
predicted  using  the  estimated  probabilities.  Use  as  an  alternative  to  explicit 
communication  (Greenstein  and  Revesman,  1986). 

Explanation:  As  discussed,  explicit  communication  between  the  operator 
and  the  aid  is  time-expensive.  Greenstein  and  Revesman  have  proposed 
a  model  by  which  explicit  operator-aid  communication  can  be  avoided.  In 
this  two-stage  model,  the  system  first  models  the  probability  that  a  human 
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will  detect  and  respond  to  an  event.  Second,  the  system  predicts  operator 
control  actions  using  the  estimated  detection  and  response  probabilities. 
This  method  allows  the  selection  of  optimal  aiding  actions  based  on 
predictions  of  humans'  actions,  and  is  considered  a  practical  alternative  to 
explicit  communication.  A  queueing  model  of  human  decision  making  has 
been  described  by  Chu  and  Rouse  (1979). 

Empirical  Support: 

4.6.1. i  When  info  must  be  updated  quickly,  the  most  important  information  should 

be  cued  to  Insure  that  these  items  are  the  first  to  be  processed  off  of  the 
sensory  register  (Barnes,  1981,  from  Sperling,  1961). 

Explanation:  This  guideline  is  often  overlooked  during  the  design  of 
information  displays  in  aiding  systems.  Barnes  states  that  cueing 
important  items  helps  insure  that  they  will  be  the  first  to  be  processed  by 
the  operator. 

Empirical  Support:  This  guideline  is  a  basic  Information  presentation 
guideline  and  was  developed  from  the  work  of  Sperling  (1961).  Sperling 
showed  that  cued  information  is  the  first  Information  processed  off  of  the 
visual  sensory  registers. 

4.6.1.)  Stress  spatial  representation  of  information  and  symbols  during  overload 
situations  involving  tracking  tasks.  Increased  performance  on  the  task 
often  results  (Barnes,  1981). 

Explanation:  In  tracking  task  scenarios,  mental  overload  may  be  reduced  if 
information  is  presented  spatially  or  graphically.  Better  task  performance 
is  likely  to  follow  reduced  levels  of  overload. 

Empirical  Support:  This  guideline  is  based  on  a  study  by  Baddeley  and 
Lieberman  (1980)  that  showed  increased  performance  of  tracking  tasks 
when  information  was  transformed  to  a  spatial  representation  format. 
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4.6.2  Implications  on  Workload 

Communication  between  operator  and  aid  can  result  in  increased  workload 
for  either  or  both  agents.  If  it  Is  explicit,  it  imposes  another  task  on  the  operator.  If 
communication  is  implicit,  however,  the  operator's  workload  may  not  be  affected 
because  the  operator  does  not  have  to  make  an  effort  to  communicate  his  actions  to 
the  aid.  Implicit  communication  is,  understandably,  more  difficult  than  explicit 
communication. 

Guidelines 

4.6.2. a  A  good  decision  aid  should  reduce  the  number  of  response  options  in  the 

cockpit.  This  research  showed  that  response  loading  factors,  and  not 
number  of  threats  were  the  bottleneck  for  a  threat  evaluation  task  (Barnes, 
1985). 

Explanation:  Workload  will  be  reduced  if  the  number  of  possible  response 
options  in  the  cockpit  is  minimized.  Barnes’  study  showed  that  the  number 
of  responses,  not  the  number  of  threats  perceived,  was  the  main  source  of 
increased  workload  in  a  threat  evaluation  task.  This  implies  that  pilot  - 
decision  aid  communication  should  be  designed  so  that  required  pilot 
responses  are  minimized. 

Empirical  Support:  Validation  is  provided  by  Barnes'  study. 

4.6.2. b  information  presentation  rate  should  not  exceed  the  short  term  memory 

(STM)  span  of  about  7  items  in  active  memory  at  once  (Barnes,  1981 ,  from 
Miller,  1956  ). 

Explanation:  If  a  system  communicates  more  than  7  items  of  information 
to  the  pilot  at  once,  the  pilot  will  experience  overload.  Designers  should 
therefore  configure  decision  aids  to  assist  decision  making  when  operators 
have  to  account  for  7  or  more  items. 

Empirical  Support:  Miller’s  work  (1956)  and  numerous  replications  of  his 
work  support  this  guideline. 
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4.6.2. c  Utilize  spatial  representations  of  aiding  information  (where  possible) 

instead  of  verbal  or  textual  displays  in  high  workload  situations  (Barnes, 
1981). 

Explanation:  In  high  workload  situations,  humans  are  better  able  to  attend 
to  spatial  representations.  Barnes  suggests,  therefore,  that  they  be  used 
wherever  possible,  particularly  in  high  workload  situations.  One  caveat  to 
this  guideline  is  not  always  easy  or  even  possible  to  create  spatial 
representations  of  information  . 

Empirical  Support:  Baddeley  and  Lieberman  (1980)  provide  support  for 
this  guideline,  showing  that  spatial  working  memory  is  used  to  store 
information  about  the  relationships  about  objects  in  space.  In  addition, 
Baddeley  and  Lieberman  have  found  that  if  information  is  presented 
spatially,  the  probability  of  developing  a  high  workload  situation  is 
reduced. 

4.6. 2. d  Model-based  communication  systems  should  be  used  in  real-time  systems 

where  the  aid’s  actions  may  parallel  operator  actions.  This  approach 
allows  the  aid  to  reduce  workload  while  maintaining  the  operator's 
perception  of  control  (Revesman  and  Greenstein,  1986). 

Explanation:  In  order  to  maintain  the  operator's  perception  of  control  while 
reducing  workload,  Revesman  and  Greenstein  suggest  using  model- 
based  communication  in  aiding  systems.  It  is  believed  that  modeling 
communication  processes  will  provide  a  valid  substitute  for  actual 
communication.  This  will  allow  the  aiding  system  to  act  in  coordination 
with  the  operator  without  interrupting  him,  increasing  workload,  or  causing 
error.  Examples  of  implicit  communication  models  have  been  discussed  in 
Guidelines  4.6.1. e,  4.6. 1.f,  and  4.6. 1.h. 

Empirical  Support:  Additional  validation  is  required  to  determine  whether 
these  models  should  be  used  in  real-time  systems. 
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4.6.2. e  Dialogue-based  communication  should  be  minimized  between  the 

operator  and  aid.  Although  it  allows  explicit  communication,  it  is  not 
optimal  because  it  may  increase  operator  workload  rather  than  decrease 
workload  through  aiding  (Revesman  and  Greenstein,  1986). 

Explanation:  Explicit  or  dialogue-based  communication  is  acceptable  only 
in  low  workload  situations  because  it  places  attentional  demands  on 
operators  and  may  increase  workload.  In  situations  with  increasing  levels 
of  workload,  implicit  (modeled)  communication  should  be  used. 

Empirical  Support:  More  validation  is  needed  to  determine  at  what  level  of 
workload  dialogue-based  communication  is  unacceptable. 

4.6.3  implications  on  Operator  Acceptance 

The  ease  with  which  operator-aid  communication  occurs  will  significantly 
affect  operator  acceptance  of  the  aid.  Availability  of  information,  Information 
bandwidth,  an  understanding  of  how  the  aid  works,  and  communication  about 
adaptation  are  the  types  of  critical  information  that  determine  whether  the  operator 
accepts  the  aid’s  assistance. 

Guidelines 

4.6.3. a  Procedural  Information,  or  information  about  rules  or  algorithms  that  the 

aid  is  using,  allows  the  operator  to  make  effective  decisions  in  situations 
where  those  rules  apply  (Morris,  Rouse  and  Ward,  1985). 

Explanation:  There  are  three  types  of  operator-aid  communication 
information  that  may  affect  user  acceptance  of  an  aid:  procedural,  process, 
and  product  information.  The  first  two  are  discussed  in  this  and  the 
following  guideline.  The  third  is  discussed  In  Section  4.6.4,  Guideline 

4.6.4. h.  Procedural  information  is  information  about  rules  or  algorithms 
used  by  the  aid.  Availability  of  this  information  allows  the  operator  to  make 
effective  decisions  In  situations  where  the  aid  is  using  such  procedures. 
Knowledge  of  procedural  information  fosters  user  acceptance  of  the  aid 
because  the  operator  is  able  to  understand  how  the  aid  is  functioning. 
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Empirical  Support:  Morris,  Rouse  and  Ward  showed  that  awareness  of 
procedural  information  increased  operator  acceptance  of  the  aid. 

4.6.3.b  Process  information  or  Information  about  how  the  aid  accomplishes  tasks 
is  necessary  when  the  operator  is  considering  the  use  of  an  aid  in 
unfamiliar  contexts  (Mom's,  Rouse  and  Ward,  1985). 

Explanation:  The  second  type  of  operator-aid  communication  information 
presented  by  Morris  et  al.  is  process  information.  This  information  includes 
how  the  aid  performs  particular  tasks.  It  was  found  that  operators  require 
process  information  when  deciding  whether  or  not  to  use  an  aid  in 
unfamiliar  situations,  or  when  trying  to  identify  the  nature  and  extent  of 
aiding  malfunctions.  Operator  acceptance  of  an  aid  hinges  on  the 
availability  of  process  information  in  new  contexts. 

Empirical  Support:  This  guideline  was  validated  in  the  Morris  et  al.  study. 

In  addition,  Andes  and  Rouse  (1991)  found  that  procedural  and  process 
information  are  important  to  designers  of  adaptive  aiding  systems.  See 
Guidelines  4.1. 1.c,  4.1. 3.a,  4.5.1. a,  and  4.5.1. b. 

4.6.3.C  Adaptation  Is  facilitated  if  the  user  and  the  aid  have  a  good  conceptual 
model  of  how  each  of  the  other  agents  work.  Improvements  in  the  user’s 
model  of  the  system  are  thought  to  be  more  beneficial  than  the  converse 
(Lehneret  al.  1987). 

Explanation:  This  implies  that  gains  in  operator  understanding  of  how  a 
system  functions  will  lead  to  increased  probability  of  acceptance.  Lehner 
et  al.  also  believe  that  improvements  in  the  operator’s  model  of  the  system 
are  more  beneficial  to  system  performance  and  operator  acceptance  than 
are  improvements  to  the  aid's  model  of  the  operator. 

Empirical  Support:  Although  this  guideline  is  present  In  several  aiding 
paradigms  (Andes,  1987;  Rouse,  Geddes  and  Curry,  1987),  it  has  not  yet 
been  empirically  tested. 


74 


Contract  No.  F33615-88-C-3612 
Report  No.  NAWCADWAR-92085-60 


4.6.3. d  Dynamic  allocation  (i.e.,  adaptive  aiding)  has  the  disadvantage  of  creating 

the  possibility  of  conflict  between  aid  and  operator  when  both  attempt  the 
same  task.  Minimize  this  possibility  by  ensuring  that  each  entity  is  aware 
of  the  other’s  actions  (Revesman  and  Greenstein,  1986). 

Explanation:  In  dynamic  allocation  situations,  the  allocation  of  tasks  to 
either  the  operator  or  the  aid  changes  over  time.  It  is  possible  and  perhaps 
likely  that  in  such  situations,  the  operator  and  the  aid  may  attempt  to 
perform  the  same  task  at  the  same  time.  It  Is  therefore  necessary  to 
Implement  explicit  communication  methods  In  aiding  systems  with  dynamic 
allocation.  This  will  enable  each  agent  to  become  aware  of  the  other’s 
actions,  and  is  likely  to  lead  to  Increased  operator  acceptance  of  the 
system.  The  type  and  amount  of  communication  in  these  situations, 
however,  must  be  optimized  so  as  not  to  increase  workload  associated 
with  communication. 

Empirical  Support:  This  guideline  has  been  espoused  by  several 
researchers  including  Rouse  (1988),  Lehner  ei  al.  (1387),  and  Andes 
(1987).  Further  research  is  necessary  to  determine  exactly  how  to 
optimize  type  and  amount  of  communications  in  dynamic  allocation 
settings. 

4.6.4  Implications  on  Situation  Assessment 

The  ability  of  the  operator  to  assess  a  given  situation  depends  on  his  ability  to 
acquire  information  about  system  status  from  the  aid.  Conversely,  the  ability  of  the 
aid  to  assess  a  situation  depends  on  its  awareness  of  the  operator’s  actions  and  the 
effects  of  those  actions.  Thus,  operator-aid  communication  also  affects  situation 
assessment  capability  of  any  system. 

Guidelines 

4.6.4. a  A  variety  of  human  factors  principles  (e.g.,  vigilance  support,  navigational 

support,  maintenance  of  user  context,  etc.)  for  the  design  of  complex 
systems  displays  apply  directly  to  the  design  of  adaptive  aiding  system 
displays.  See  Noah  and  Halpin  (1986)  for  list  (Rouse,  1988). 
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Explanation:  Lessons  learned  from  researchers  in  the  display  design 
community  can  be  applied  to  the  design  of  adaptive  aiding  systems.  In 
particular,  adherence  to  such  guidelines  in  aiding  systems  information 
displays  may  facilitate  operator-aid  communication.  This  may  ultimately 
increase  the  ability  of  the  operator  to  make  situation  assessment  decisions. 

Empirical  Support:  Noah  and  Halpin  (1986)  have  addressed  display 

design  with  regard  to  communicating  complex  Information.  Furthermore, 

Barnes  (1981)  has  summarized  information  display  guidelines  that  are 

useful  in  the  display  of  aiding  information. 

\ 

4.6.4. b  Factors  that  affect  display  of  aiding  information  include  mode  of  information 

presentation,  type  of  information  presented,  timeliness  of  delivery,  and 
depth  of  aiding  (amount  of  information)  (Andes,  1990). 

Explanation:  Several  factors  may  affect  the  pilot's  ability  to  assess 
information  In  an  aiding  display.  The  designer  should  consider  these 
factors  during  the  design  of  the  pilot-aid  task  allocation  interface  because 
the  nature  of  this  interface  will  affect  the  ability  of  the  pilot  to  assess  the 
state  of  the  information  presented  to  him. 

Empirical  Support: 

4.6.4. C  System  information  should  be  reorganized  and  displayed  in  either 

aggregated  or  disaggregated  patterns  depending  on  the  state  of  the 
system.  Use  aggregated  information  displays  for  overall  system  situation 
awareness;  use  disaggregated  displays  to  highlight  errors  or  detailed 
information  (Forester,  1986). 

Explanation:  This  guideline  addresses  the  presentation  of  system-relevant 
information  to  facilitate  operator  situation  assessment  ability.  Forester 
suggests  that  designers  consider  possible  states'of  the  system  and  display 
information  in  either  an  aggregated  or  disaggregated  pattern  accordingly. 
Aggregated  information  displays  should  be  used  to  display  system 
situation  Information  because  such  displays  allow  the  operator  to  assess 
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the  overall  state  of  the  system.  Disaggregated  displays  draw  operator 
attention  to  system  errors  and  other  detailed  information. 

Empirical  Support:  Forester  validated  this  guideline  in  three  experiments, 
showing  that  both  aided  and  unaided  performance  increased  when 
display  modality  was  in  accord  with  the  state  of  the  system. 

4.6.4. d  Avoid  the  application  of  a  wide-spectrum  (i.e.,  multi-capability)  pilot  aid 

because  either  the  pilot  or  the  aid  can  become  confused  as  to  their  proper 
duties  at  a  particular  time  (Krobusek  et  al.  1989). 

Explanation:  A  wide-spectrum  pilot  aid  may  result  In  unclearly  specified 
task  allocation.  This  would  limit  the  ability  of  the  pilot  to  perform  well 
because  he  would  not  be  certain  upon  which  tasks  to  focus  his  attention. 
Furthermore,  since  a  wide-spectrum  aid  would  cause  the  pilot  to  focus 
attention  on  task  allocation,  he  would  not  be  able  to  sufficiently  assess  the 
state  of  the  system  or  the  environment. 

Empirical  Support:  This  guideline  is  consistent  with  the  findings  of  Lehner, 
Cohen,  Mullin,  Thompson,  and  Laskey  (1987),  Lind  (1989),  and  Moss, 
Reising  and  Hudson  (1984). 

4.6.4. e  Interaction  with  the  aid  must  be  time  and  effort  efficient.  The  pilot  must  be 

able  to  readily  determine  system  status  and  allocation  of  task  between  pilot 
and  aid  at  a  glance  (Krobusek  et  al.  1989). 

Explanation:  Time  and  effort  efficient  situation  assessment  can  be 
achieved  if  the  pilot  is  capable  of  readily  determining  system  status  and 
allocation  of  tasks  between  himself  and  the  aid. 

Empirical  Support:  This  guideline  has  been  addressed  in  the  Pilot’s 
Associate  literature,  but  still  requires  empirical  validation. 

4.6.4. f  The  aid  must  keep  track  of  tasks  currently  being  executed,  task  contexts 

that  are  no  longer  valid,  and  provide  a  mechanism  for  keeping  the  operator 
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abreast  of  its  activities,  for  example,  cross-hatch  overlays  of  menus  and 
termination  conditions  in  scripts  (Andes,  1987). 

Explanation:  An  aid  that  is  able  to  successfully  supply  situation 
assessment  information  to  the  operator  must  perform  several  functions. 
Andes  states  that  such  an  aid  must  keep  track  of  current  tasks  and  invalid 
task  contexts,  and  must  be  able  to  communicate  its  activities  to  the 
operator.  Cross-hatching  menus  are  a  way  to  notify  the  operator  that 
particular  menus  are  no  longer  applicable  to  a  situation;  termination 
conditions  in  system  programs  are  a  way  for  the  system  to  halt  current 
tasks  and  to  notify  the  operator  of  such  activities. 

Empirical  Support:  Although  several  approaches  to  communicating  this 
type  of  situation  awareness  have  been  proposed,  the  lack  of  operational 
systems  has  made  the  validation  of  this  guideline  relatively  infeasible. 

4.6.4. g  During  design,  attempt  to  anticipate  situations  in  which  the  operator  may 

have  to  intervene  in  automated  tasks.  Use  this  information  in  the  design  of 
early  warning  notification  displays  for  aid  failure  and/or  performance 
decrements  (Morrison  et  al.  1990). 

Explanation:  Morrison  et  al.  suggest  that  designers  should  determine  In 
which  situations  aid  failure  would  require  operator  Intervention  and  should 
estimate  the  time  required  for  operators  to  adjust  to  ensuing  task  load 
changes.  Although  such  time  estimations  may  be  variable,  designers 
should  use  these  estimations  to  indicate  how  to  implement  early  warning 
notification  displays.  Designers'  estimations  of  pilot  situation  assessment 
ability  are  critical  to  pilot  and  mission  survival  during  system  failure. 

Empirical  Support:  The  ability  of  designers  to  adhere  to  this  guideline  has 
not  yet  been  observed. 

4.6.4, h  Product  information  -  Information  about  normal  aiding  system  output  -  is 

necessary  to  allow  the  operator  to  determine  whether  the  system  is 
functioning  properly  (Morris,  Rouse  and  Ward,  1985). 
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Explanation:  Product  information  is  the  third  type  of  operator-aid 
communication  information.  (See  Guidelines  4.6.3.a  and  4.6.3.b  for 
explanations  of  procedural  and  process  information,  respectively.)  Product 
Information  addresses  normal  aiding  system  output  and  is  required  for  the 
operator  to  validate  proper  system  functioning.  This  is  expected  as  a  basic 
function  expected  of  aiding  systems.  Furthermore,  providing  product 
Information  to  the  operator  would  not  be  difficult  because  such  information 
is  readily  available  to  the  aid. 

Empirical  Support:  Morris,  Rouse  and  Ward  showed  that  awareness  of 
product  Information  increased  operator  ability  to  assess  the  state  of  the 
system  and  the  external  situation. 

4.6.4. i  Timeliness  of  information  presentation  -  If  information  is  static  (acquired 

before  the  decision  point),  it  can  be  readily  applied.  If  Information  source  is 
dynamic  (i.e.,  system  feedback),  it  must  be  presented  on-line  to  ensure  a 
timely  decision  (Morris,  Rouse  and  Ward,  1985). 

Explanation:  Static  information  is  acquired  before  critical  decision  points 
and  can  be  readily  applied  when  necessary.  Dynamic  information  (such 
as  system  feedback)  differs  in  that  it  changes  over  time,  and  this  requires 
that  it  is  presented  on-line  and  on-demand.  Such  on-line  presentation  is 
necessary  to  allow  the  operator  to  make  timely  decisions  with  the  most 
current  system  information.  Using  static  information  may  lead  to  incorrect 
decisions  because  it  does  not  account  for  the  most  current  state  of  the 
system.  Successful  incorporation  of  static  and  dynamic  information  within 
operator-aid  communication  will  facilitate  the  ability  of  the  operator  to 
maintain  acceptable  levels  of  situation  assessment. 

Empirical  Support:  An  information  requirements  analysis  must  be 
conducted  during  design  to  determine  how  much  information  should  be 
presented  to  the  operator  in  a  given  situation  (Andes  and  Rouse,  1991). 

4.6.4.j  As  the  level  of  familiarity  changes,  procedural  information  becomes  less 
applicable,  and  more  product  and  process  information  is  needed.  The 
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value  of  on-line  information  over  off-line  information  also  increases  relative 
to  the  type  of  information  required  (Morris  and  Rouse,  1985). 

Explanation:  Morris  and  Rouse  (1986)  have  formed  a  general  principle  of 
information  requirements  in  aiding  from  research  addressing  such 
requirements:  "As  the  nature  of  the  situation  changes  from  familiar  to 
unfamiliar,  and/or  the  performance  of  the  aid  degrades,  there  is  a 
corresponding  change  in  the  information  requirements  of  the  operator. 

The  direction  of  the  change  is  toward  more  static  information  with 
decreased  situation  familiarity,  and  toward  more  dynamic  information  with 
Increased  situation  familiarity."  Thus,  as  an  operator  becomes  more 
familiar  with  a  given  situation,  he  requires  less  procedural  information  and 
more  product  and  process  Information.  This  relates  to  a  decreased  value 
of  off-line  information  and  an  increased  value  of  on-line  information,  and 
implies  that  the  operator's  ability  to  use  dynamic  information  improves  with 
increases  in  situation  familiarity. 

Empirical  Support:  This  guideline  and  the  general  hypothesis  about 
information  requirements  need  further  validation. 

4.6.4.k  Immediate  feedback  is  necessary  from  command  and  control  orders  to 

ensure  successful  situation  assessment  of  aiding  (Morris  and  Zee,  1988). 

Explanation:  Information  that  is  critical  to  accurate  situation  assessment  is 
feedback  from  command  and  control  (C2)  orders.  In  C2  situations,  the 
decision  maker  must  have  the  ability  to  monitor  the  activities  of  others  and 
determine  whether  his  orders  have  been  properly  executed.  It  is  also 
important  for  the  decision  maker  to  easily  determine  whether  the  aid  has 
responded  to  his  commands.  Such  information  is  required  by  the  decision 
maker  to  make  further  C2  actions  and  to  properly  assess  what  the  system 
as  a  whole  is  doing. 

Empirical  Support:  Experimentation  is  necessary  to  validate  this  guideline. 
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4.6:  Section  Summary 

This  section  discussed  the  design  of  aiding  systems  with  regard  to  the  nature 
of  communication  between  the  aiding  system  and  the  operator.  These  guidelines 
suggest  that  designers  consider  the  costs  of  explicit  versus  Implicit  communication 
and  the  appropriateness  of  the  type  of  communication  to  the  situation.  Further, 
designers  should  consider  the  effects  of  whether  an  operator  chooses  to  heed  or 
ignore  the  aid’s  advice.  Designers  should  also  allow  model*based  communciation, 
and  minimize  displayed  response  options.  These  guidelines  suggest  that  designers 
should:  a)  design  aiding  systems  with  the  capacity  to  display  variable  information 
presentation  formats,  to  cue  information,  and  to  allow  the  spatial  representation  of 
information,  and  b)  design  aiding  system  displays  to  promote  operator/aid  time  and 
effort  efficiency. 


5.0  Summary  and  discussion 

A  large  number  of  sources  were  reviewed  in  the  preparation  of  this 
document.  Research  from  systems  engineering,  artificial  intelligence,  human 
factors  psychology,  and  human  performance  in  complex  systems  was  reviewed 
in  an  effort  to  cover  all  facets  of  the  design  of  adaptive  aiding  systems.  This 
review  includes  literature  beginning  with  Rouse's  (1975)  paper  and  continuing 
through  adaptive  aiding  research  results  to  the  present. 

A  total  of  142  guidelines  for  adaptive  aiding  system  design  were 
compiled  from  over  40  references.  Although  each  source  contributed 
significantly  to  the  number  and  quality  of  the  guidelines,  some  sources  had  a 
large  influence  on  the  way  the  guidelines  evolved.  Sen  (1984),  for  example, 
used  decision  theory  to  provide  insight  into  the  nature  of  operator  /  aid 
interaction.  Likewise,  Weisbrod,  Davis  and  Freedy  (1977)  and  Revesman  and 
Greenstein  (1986)  incorporated  decision  theory  into  specifications  for  decision 
support  systems.  Krobusek,  Boys  and  Palko  (1989)  provided  significant  input 
on  the  performance  of  aiding  systems  and  on  training  requirements  for  such 
systems.  Furthermore,  Morris,  Rouse  and  Ward  (1985)  and  Morris  and  Rouse 
(1 986)  provided  an  account  of  user  attitudes  toward  an  aid,  as  well  as  user 
interaction  tendencies.  Finally,  from  a  practical  point  of  view,  Norcio  and 
Stanley  (1989)  discussed  how  adaptive  interfaces  may  benefit  users  of 
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adaptive  aiding  systems,  and  Andes  and  Rouse  (1991)  showed  what 
information  designers  value  during  the  development  of  aiding  systems. 

The  guideline  taxonomy  constructed  from  Rouse’s  framework  for  design 
appears  to  address  all  issues  concerning  adaptive  aid  design.  It  serves  three 
major  purposes  in  the  current  review  context:  First,  it  provides  quick  reference 
structuring  for  immediate  access  to  specific  design  information  and  cross 
references  the  information  on  projected  implications  of  the  information  on  aid 
design.  Second,  it  highlights  where  most  of  the  research  to  date  has  been 
focused  and  shows  where  research  is  needed  to  fill  out  the  concept.  Finally,  it 
provides  a  robust  structure  for  future  formulation  of  design  guidelines.  Table  2 
provides  a  statistical  summary  of  the  guidelines  in  this  document.  -  There  are  two 
numbers  in  each  cell  of  the  table:  the  first  represents  the  number  of  guidelines 
that  apply  to  that  area  of  the  taxonomy,  and  the  second  represents  the  number 
of  guidelines  with  empirical  support. 


DESIGN  FRAMEWORK 

USER 

SITUATION 

QUESTION 

PERFORMANCE 

WORKLOAD 

ACCEPTANCE 

ASSESSMENT 

What  is  adapted  to? 

17/8 

10/5 

8/3 

3/2 

Who  does  the  adapting? 

10/7 

3/2 

4/4 

3/0 

When  does  the  adaptation 
occur? 

7/4 

5/4 

5/2 

1/0 

Whal  methods  of  adaptation 
apply? 

9/1 

1/1 

1/0 

0 

How  is  adaptatbn  done? 

14/7 

5/1 

5/0 

1/0 

1  0/4 

5/3 

4/3 

11  /5 

TOTALS 

67/31 

29/16 

27/12 

19/7 

Table  2  -  Design  Guideline  Classification  Taxonomy 

Generally,  the  taxonomy  served  as  a  useful  organizational  tool  for  design 
guidelines.  We  may  desire  to  augment  the  framework  in  the  future,  specifically 
in  the  implications  area.  Possibly,  implications  on  implementation  difficulty 
should  be  added  when  more  systems  are  deployed  and  lessons  learned  are 
compiled.  Basically,  we  will  be  producing  guidelines  about  better  ways  to 
implement  aid  functionality. 
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Although  quite  a  few  useful  design  guidelines  were  extracted  from  the 
literature,  one  can  see  that  most  of  the  guidelines  are  concerned  with  what 
agent  or  task  is  adapted  to,  and  how  that  adaptation  should  be  accomplished. 
Specifically,  a  large  number  of  guidelines  addressed  various  issues  in  terms  of 
resulting  system  performance.  This  is  a  logical  observation,  because  the 
motivation  behind  adaptive  aiding  technology  is  to  Increase  overall  system 
performance  by  augmenting  the  operator's  performance  through  task 
assistance  and  workload  reduction.  However,  the  other  two  affect  areas  (user 
acceptance  and  situation  assessment)  are  in  dire  need  of  research  support. 

Applicability  of  types  of  aiding  is  another  area  in  need  of  research 
support.  The  "meta-design"  aiding  study  conducted  by  Andes  and  Rouse 
(1991)  showed  that  designers  are  very  interested  in  the  interaction  between 
types  of  aiding  and  applicability  of  these  aiding  types.  Since  there  are  not  many 
implemented  systems  or  studies  analyzing  different  modes  of  aiding  in  the 
same  task  environment,  it  Is  expected  that  few  guidelines  would  be  extracted. 
This  is  a  prime  area  for  future  research,  however.  Without  such  Information,  the 
relative  applicability  of  different  types  of  aiding  will  still  be  In  question.  Further, 
as  adaptive  aid  design  becomes  more  of  an  engineering  process,  rather  than 
art,  there  is  a  need  for  aid  functional  consistency.  The  interaction  characteristics 
between  operator  and  aid  should  become  more  consistent  within  specific  (I.e., 
aircraft  cockpits)  domains. 

More  investigation  into  the  design  philosophy  underlying  the  resulting 
aiding  system  also  needs  to  be  conducted.  As  can  be  seen  in  the  number  of 
guidelines  pertaining  to  the  automation  approach  taken  and  functions  to  be 
allocated,  the  philosophy  of  aiding  is  the  most  important  consideration  in 
design.  There  is  a  need,  however,  tq  develop  specific  guidelines  about  when 
an  automation  approach  is  more  applicable  than  another. 

Finally,  less  emphasis  needs  to  be  placed  on  models  of  human 
performance  and  resulting  guidelines.  Rather,  more  emphasis  should  be 
placed  on  how  the  operator  views  the  partnership  between  operator  and  aid.  In 
particular,  development  of  models  that  consider  operator  task  loading,  internal 
psychological  factors,  and  then  system  performance  implications  on  aiding 
ought  to  be  developed. 
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6.0  Future  directions 

This  document  is  the  second  step  in  the  production  of  a  comprehensive 
reference  of  adaptive  aiding  design  guidelines.  The  aiding  systems  designer,  as 
stated  early  on  in  this  document,  must  be  familiar  with  several  diverse  fields  of 
research:  human  factors  psychology,  software  engineering,  human  performance, 
etc.  It  Is  unreasonable  to  assume  that  the  systems  designer  would  be  intimately 
familiar  with  all  related  fields.  Instead,  it  is  the  adaptive  aiding  researcher's 
responsibility  to  educate  potential  designers  about  sources  of  information  and 
directions  on  how  to  use  that  Information  during  the  design  process. 

Towards  that  end,  it  appears  that  the  taxonomy  developed  during  this  review 
will  serve  as  a  reasonable  framework  for  the  expanding  guidelines  reference.  In 
support  of  this,  there  are  a  few  recommendations  for  future  work  In  compiling  the 
adaptive  aiding  design  guidelines.  First,  it  is  essential  to  Identify  which  areas  are 
lacking  significant  design  direction.  As  stated  in  the  summary  of  this  document,  the 
areas  most  in  need  of  useful  guidelines  are  those  concerning  user  acceptance  of 
aiding  and  situation  assessment  in  the  presence  of  an  aid.  Future  work  should 
emphasize  these  areas.  One  couid  conclude  that  the  operator-aid  interaction  is  Itifi 
area  in  need  of  guideline  fortification.  In  addition,  more  concrete  guidelines  should 
be  produced:  researchers  typically  do  not  report  results  in  a  format  Intended  for 
designers.  With  the  implementation  of  more  aiding  systems,  we  should  see 
emphasis  being  placed  on  the  design  process  and  therefore  guidelines  with  well 
defined  application  areas  should  result. 

Second,  the  breadth  of  sources  should  be  expanded  in  the  next  iteration.  In 
particular,  implemented  systems  literature  should  be  Included  to  extract  design 
guidelines  from  lessons  learned  during  Implementation  and  evaluation  of  the 
systems.  Other  reference  sources  (i.e.,  different  databases,  etc.)  should  be  reviewed 
to  identify  useful  information  that  was  not  uncovered  in  this  pass.  Accident  and 
safety  analysis  Information  will  be  invaluable  in  fleshing  out  the  guideline  framework. 

Finally,  related  pure  and  applied  research  from  other  research  areas  should 
be  reviewed  and  included  in  the  next  iteration  of  the  design  guidelines.  Although 
this  theme  was  addressed  in  the  current  review,  research  results  that  were 
peripheral  to  the  adaptive  aiding  concept  (i.e.,  results  that  have  bearing  on,  but  are 
not  directly  produced  from  an  aiding  environment)  were  not  stressed.  There  is  a 
wealth  of  research  results  that  could  be  interpreted  for  application  to  adaptive  aiding 
design  problems  (e.g.,  vigilance  studies,  group  decision  making  studies, 
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collaborative  work  analyses,  etc.).  The  major  work  involved  on  this  theme  will 
concern  proper  interpretation  of  the  results  and  data  for  aid  design  application. 
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Glossary  of  Terms 


adaptive  aiding  •  A  systems  automation  philosophy  that  proposes  the  use  of 
automation  to  assist  the  operator  only  when  system  performance  is  likely  to 
degrade  past  the  point  of  acceptability  in  the  near  future  (Rouse  and  Rouse, 
1983). 

adaptive  automation  -  Another  term  for  adaptive  aiding.  Stresses  adaptability  of 
automation  in  terms  of  the  regulation  of  operator  workload  and  vigilance, 
maintenance  of  skill  levels,  and  task  involvement  at  both  cognitive  and  manual 
perforrnance  levels. 

adaptive  function  allocation  -  Alternate  term  for  adaptive  aiding  and  adaptive 
automation.  Roots  founded  in  functional  task  allocation  perspective  of  Fitts  • 
(1951). 

adaptive  pattern  classification  techniques  -  Founded  primarily  in  the  statistical 
domain,  adaptive  pattern  classification  techniques  range  from  n-dimensional 
mappings  of  data  to  groups  to  complex  pattern  recognizers  (e.g.,  neural 
networks).  These  techniques  are  applicable  when  the  state  space  contains 
multi-dimensional  input,  output,  and  state  vectors,  and  especially  when  normal 
(i.e.,  linear  models)  statistical  analysis  techniques  do  not  apply. 

aid-initiated  intervention  -  A  type  of  Index  of  aiding  system  adaptation.  In  this 
case,  aiding  is  activated  by  exceeding  a  pre-defined  measurement  (i.e., 
performance)  or  modeling  (i.e.,  operator  workload)  threshold  (Rouse,  1988). 

cognitive  style  -  The  characteristic  manner  in  which  an  individual  perceives, 
thinks  about,  and  reacts  to  the  environment.  This  term  is  commonly  seen  in  the 
context  of  individual  differences  (Hunt,  1982). 

concept  development  -  The  process  by  which  individuals  acquire  and 
parameterize  new  category  terms  for  understanding  of  the  perceived  world 
(Neisser,  1987). 

conscious  overload  avoidance  strategy  -  The  automatic  "task  shedding" 
behavior  exhibited  by  human  operators  in  high  workload  multi-task 
environments.  This  behavior  is  often  observed  during  the  period  before 
operator  task  overload  when  the  operator  anticipates  imminent  resource 
overload  (Sen,  1984). 

context  customization  -  A  characteristic  of  adaptive  user  interfaces.  Context 
customization  features  allow  the  user  to  personalize  the  aiding  system's  display 
presentation,  create  new  links  between  system  parameters,  and  vary  the  depth 
of  information  presented  to  the  user  based  on  user  expertise  (Noah  and  Halpin, 
1989). 
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critical  event  logic  -  A  method  of  automation  implementation  by  which  activation 
of  the  automation  is  tied  to  the  occurrence  of  specific  tactical  events  (Barnes, 
1985). 

decision  maker's  dominance  structure  -  In  decision  theory,  this  structure  is 
represented  by  the  fact  that  a  decision  alternative  category,  x,  is  preferred 
("dominates")  over  another  decision  alternative  category,  y.  That  is:  xi  >=  yi ; 
for  all  decision  instances,  i.  This  approach  is  taken  to  account  for  all  decision 
alternative  classes  within  context.  In  a  dominance  structure,  the  idea  of 
dominance  exploits  only  the  ordinal  character  of  the  decision  alternatives  (i.e., 
simple  preference  of  x  over  y),  not  the  cardinal  character  of  the  alternatives  (i.e., 
that  X  is  3  greater  than  y)  (Keeney  and  Raiffa,  1976). 

decision  analysis  *  The  process  of  determining  either.(or  both)  probabilistic  or 
utilitarian  judgments  that  are  produced  in  response  to  special  circumstances 
yielding  alternative  choices  to  a  decision  maker  (Hammond,  McClelland,  and 
Mumpower,  1980). 

depth  of  information  -  Detail  of  aiding  status  information  supplied  to  the  operator 
about  the  aid  (see  process,  product,  and  procedural  information  definitions  for 
comments)  (Andes,  1990). 

direct  assessment  -  Refers  to  the  employment  of  psychophysical  measurements 
to  estimate  operator  task  and  workload  in  determining  the  need  for  aiding 
(Parasuraman  et  al.  1990). 

direct  judgment  •  From  Behavioral  Decision  Theory  (BDT)  (Edwards,  1962).  A 
procedure  used  to  estimate  the  parameters  of  models  of  decision  makers’ 
decision  process.  BDT  frequently  requires  the  decision  maker  to  specify  directly 
the  relative  importance  of  various  attributes  of  the  decision  problem  (e.g.,  by 
assigning  numbers  to  them  in  proportion  to  their  relative  importance). 

dynamic  utility  estimation  -  Statistical  methods  in  decision  analysis  designed  to 
produce  probabilistic  estimates  of  an  individual's  estimate  for  each  alternative 
decision  within  a  set  of  decision  alternatives  (Keeney  and  Raiffa,  1976). 

dynamic  task  allocation  -  An  approach  to  adaptive  aiding  invocation  by  which 
active  and  pending  tasks  are  assigned  to  the  operator  or  the  aid  according  to 
changing  system  state,  operator  state,  or  a  combination  of  both  (Rouse,  1975). 

dynamic  assessment  -  Refers  to  dynamic  psychophysiological  estimation  of 
operator  workload  based  on  factors  such  as  EEG,  pupillary  dilation,  ERP,  etc. 
(Parasuraman,  et.  al.,  1990). 

dynamic  workload  -  Determination  of  changing  operator  wbrkload  levels  via 
workload  estimation  techniques  (e.g.,  SWAT) . 
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embedded  training  -  A  type  of  automated  training  system  that  Is  a  component  of 
the  aiding  system  itself.  Embedded  training  allows  the  system  to  present  the 
operator  with  training  problems  or  exercises  with  preset  scenarios  using  the 
actual  aid  displays  and  controls.  Can  be  activated  during  lulls  in  system  control 
requirements  or  off-line. 

error  recognition  -  Various  artificial  intelligence  and  pattern  recognition 
techniques  for  Identifying  error  states  or  possible  upcoming  errors  based  on 
system  state  and  operator  actions.  Error  recognition  has  been  addressed  in  the 
Pilot's  Associate  system  (Rouse,  Geddes,  and  Curry,  1987). 

environment  representation  (graphical)  -  In  the  context  of  adaptive  aiding 
design  guidelines,  a  pictographic  or  graphical  representation  of  environment 
information  (e.g.,  situation  assessment)  often  enhances  the  operator's  ability  to 
assimilate  large  amounts  of  information  at  a  glance  (Morris,  Rouse,  and  Zee, 
1987). 

explicit  communication  -  Direct,  purposeful  communication  between  the  aid  and 
the  operator  concerning  activities,  awareness,  and  intentions  of  each  party.  Is 
minimally  ambiguous  but  can  impose  substantial  overhead  and  can  potentially 
exceed  the  benefits  of  aiding  (Rouse,  1988). 

framework  for  adaptive  aid  design  -  A  set  of  conceptual  design  Issues 
addressing  the  systematic  conceptual  design  of  adaptive  aiding  proposed  by 
Rouse,  1988.  The  designer  pursues  answers  to  the  set  of  structured  design 
questions  aimed  at  forming  a  conceptual  framework  for  the  aid  to  be  designed. 
While  it  is  not  possible  to  provide  generic,  context-free  answers  to  the 
questions,  it  is  possible  to  outline  the  range  of  alternative  answers  and  suggest 
principles  of  adaptation  and  Interaction  that  may  assist  designers  in  choosing 
among  the  alternatives.  This  framework  also  provides  the  fundamental 
structure  for  the  current  listing  of  design  guidelines.  See  Section  3.0  for  further 
explanation. 

function  aiding  -  After  Fitts  (1951).  This  general  automation  design  approach 
seeks  to  first  identify  those  tasks  that  are  best  suited  to  automation,  then 
introduce  automation  to  address  the  task  based  on  the  functions  necessary  to 
complete  that  task.  Also  espoused  in  the  work  of  Lind  (1989)  and  Krobusek  et 
al.  (1989). 

hot  potato  -  A  possible  effect  of  aid-initiated  task  re-allocation  in  adaptive 
aiding.  In  this  phenomenon,  the  system  has  adapted  to  the  point  where 
performance  is  acceptable  on  all  tasks  with  which  the  human  is  Involved.  Since 
performance  Is  now  acceptable,  the  aid  could  possibly  return  execution 
responsibility  of  a  partially  completed  task  back  to  the  operator,  resulting  in  an 
increase  in  operator  workload.  This  potential  dilemma  is  inconsistent  with  the 
operator-ln-charge  philosophy,  as  increased  workload,  degraded  performance, 
and  confusion  could  result  (Rouse,  Geddes,  and  Curry,  1987). 
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human-performance-centered  -  An  approach  to  systems  automation 
emphasizing  support  of  operator  manual  tasks  (e.g.,  tracking). 

human-requirements-centered  -  An  archetypal  approach  to  systems  automation 
emphasizing  support  of  the  human  system  operator  over  satisfaction  of  task 
requirements.  Contrast  with  task-requirements-centered  (Andes  and  Rouse, 
1991). 

incomplete  decision  responses  -  A  result  of  "conscious  overload  avoidance 
strategy."  See  above.  In  this  context,  decision  makers  provide  partial  decisions 
in  the  decision  space  because  they  have  only  considered  certain  segments  of 
information  necessary  to  provide  a  complete  decision  response.  After  Sen 
(1984). 

indirect  assessment  -  A  type  of  performance  or  workload  determination  used 
when  direct  assessment  is  not  available.  For  example,  in  Morris  and  Rouse 
(1986),  a  secondary  measure  of  performance,  tracking  response  latency,  was 
an  indirect  assessment  of  visual  overload. 

inference  from  simple  gamble  -  Used  for  dynamically  assessing  the  decision 
maker's  utility  functions.  Pairwise  preferences  for  decision  alternatives  are 
presented  to  the  decision  maker,  and  the  preference  values  are  stored  in  a 
database  for  later  reference  by  the  aiding  system  (Sage  and  White,  1984). 

information  processing  modality  -  A  channel  of  information  transmission  to  the 
human  operator.  Wickens  (1984)  has  written  about  modality  selection  from  the 
Multiple  Resource  Theory  perspective  (i.e.,  visual  vs.  auditory  channel 
Information  processing). 

Intent  inferencing  -  The  determination  of  a  human  operator's  motive-driven 
behavior  from  the  observation  of  overt  control  and  monitoring  actions  (Geddes, 
1989). 

knowledge  representation  scheme  -  Any  of  a  number  of  computer  data 
structure  representations  of  human  knowledge  to  be  embedded  in  an 
application.  For  example,  scripts,  plans,  goals,  and  actions  (Schank  and 
Abelson,  1981)  and  coded  representations  of  schemata  are  knowledge 
representation  schemes. 

learning  acceleration  -  Increasing  the  familiarization  and  proficiency  of  the  use 
of  aided  systems  through  concentrated  interaction  sessions  with  the  aid.  Can 
be  specifically  seen  in  embedded  training  systems,  where  training  sessions  can 
be  run  in  times  of  low  operator  interaction  requirements  (Norcio  and  Stanley, 
1989). 

levels  of  autonomy  (LOA)  -  From  Krobusek  et  al.  (1989).  IritheLOA 
methodology,  automation  is  introduced  according  to  discrete  levels:  1)  no 
Intervention:  2)  aid  automatically  executes  tasks;  3)  aid  reminds  pilot  of 
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performed  tasks;  4)  aid  prompts  pilot  about  important  tasks;  5)  aid  performs 
tasks  when  pre-specified  conditions  are  met. 

maintenance  of  user  context  *  In  the  transformation  mode  of  adaptive  aiding, 
this  procedure  is  used  to  sustain  the  user's  perception  of  the  same  tasks  and 
environment.  Often,  when  a  task  is  transformed  to  make  it  easier,  the  user  may 
perceive  two  completely  different  tasks(i.e.,  original  and  transformed  task).  This 
may  cause  context  switching  problems  and  confusion  for  the  user  (Andes, 
1990). 

mathematically  ideal  decision  maker  -  Theoretical  decision  maker  that 
considers  all  relevant  input  signals  regardless  of  time  pressure,  stress,  or  partial 
information  content  (Sen,  1984). 

mixed  dialogue  initiative  -  A  design  principle  of  aiding  system  interfaces  that 
attempts  to  reduce  the  cognitive  load  on  the  user  by  sharing  the  initiative  for 
dialog.  It  is  primarily  goal-driven,  template-based  behavior  that  takes  action  as 
appropriate  for  user  support  or  for  eliciting  behavior  from  the  user.  The  system 
can  also  volunteer  information  when  necessary.  From  Noah  and  Halpin  (1986). 

multiple  alternative  decision  -  From  decision  theory,  this  type  of  decision 
contains  a  number  of  alternative  choices  which  are  subjectively  weighted  by  the 
decision  maker's  utility  estimation  functions  (Keeney  and  Raiffa,  1976). 

mission  mode  tailoring  -  An  alternative  approach  to  adaptive  aiding  in  which  the 
particular  functions  to  be  aided  are  identified  by  which  mission  segment,  or 
"mode",  that  the  system  is  currently  engaged  (e.g.,  ingress,  attack,  air  escape, 
etc.)  (Und,  1989). 

model-based  communication  -  A  component  of  implicit  communication  by  which 
models  of  performance,  intention,  and/or  task  loading  are  relied  on  for 
expressing  the  current  operator  state  to  the  aid.  This  approach  can  greatly 
reduce  system  overhead  but  suffers  from  greater  ambiguity  regarding  the 
actions  and  intentions  of  each  party  (Revesman  and  Greenstein,  1986). 

Modified  Petri  nets  -  A  directed  multigraph  composed  of  nodes  and  arcs  that  are 
partitioned  into  two  sets,  places  and  transitions.  Each  arc  is  directed  from  an 
element  in  one  set  (place  or  transition)  to  an  element  of  the  other  set.  The  nets 
were  designed  for  modeling  of  systems  with  independent  components  (e.g., 
physical  systems,  social  systems,  computer  hardware  and  software,  etc.).  They 
are  used  to  model  the  occurrence  of  various  events  and  activities  in  the  system, 
particularly  to  model  the  flow  of  information  or  other  resources  in  the  system 
(Peterson,  1981). 

navigational  support  -  Often  associated  with  hypertext  interface  systems,  this 
type  of  user  aiding  assists  the  operator  in  the  traversal  of  embedded  information 
structure  from  within  the  system.  Examples  of  navigational  support  include 
"return  to  top  of  stack"  buttons,  and  graphical  mapping  of  user  traversal  from 
within  system. 
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operator  tailoring  -  A  process  by  which  the  system  operator  Is  allowed  to 
customize  the  aiding  interface  and  intervention  threshold{s)  according  to 
personal  preference  (Andes,  1990). 

progressive  disclosure  -  An  information  presentation  technique  by  which  the 
system  initially  provides  only  summary  information  (i.e.,  maximum  bandwidth  of 
information),  and  will  supply  detailed  information  about  the, summary  when 
queried  by  the  operator  (Noah  and  Halpin,  1986). 

performance  hysteresis  -  From  Andes  (1990).  This  phenomenon  is  a  particular 
manifestation  of  the  "hot  potato"  phenomenon.  It  is  an  impedance  to  smooth 
aid-operator  interaction,  where  cyclic  introduction  and  removal  of  aiding  is 
observed  based  on  changing  operator  performance.  The  operator's 
performance  characteristics  may  exhibit  local  maxima,  but  the  general 
performance  trend  Is  downward. 

planning  and  commitment  logic  -  One  of  the  four  general  user-system  tasks 
Identified  by  Rouse  and  Rouse  (1983).  This  logic  Is  used  by  the  operator  in 
generating,  evaluating,  and  selecting  alternative  courses  of  action  within  a 
control  system. 

procedural  information  -  A  primary  type  of  aid  status  information.  Refers  to 
information  pertaining  to  when  to  use  the  aid,  or  for  determining  intervention 
thresholds  (Morris,  Rouse,  and  Ward,  1 985). 

process  Information  -  A  primary  type  of  aid  status  information.  Refers  to 
functional  information  about  the  aid;  information  about  the  process  by  which  the 
aid  accomplishes  its  tasks.  This  information  may  allow  the  operator  to 
determine  the  applicability  of  the  aid  to  the  current  situation  (Morris,  Rouse,  and 
Ward,  1985). 

product  information  -  A  primary  type  of  aid  status  information.  Refers  to 
information  about  normal  aiding  system  output  that  allows  the  operator  to 
determine  whether  the  system  is  functioning  properly  (Morris,  Rouse,  and  Ward, 
1985). 

proxy  measures  of  primary  indices  of  concern  -  Substitute  measurements  used 
in  the  determination  of  the  need  for  aiding.  These  secondary  indices  (e.g.,  task 
percentage  completion)  can  be  used  to  estimate  primary  indices  of  operator 
performance  when  the  primary  quantity  cannot  be  ascertained.  Example  is  the 
use  of  operator  intent  inference  in  lieu  of  the  operator  explicitly  indicating  his 
intentions  to  the  aid  (Morris,  Rouse,  Ward,  and  Frey,  1984). 

principles  of  adaptation  -  General  design  principles  concerning  when  and  how 
adaptive  aiding  applies,  as  well  as  the  underlying  mechanisms  of  adaptation 
(Rouse,  1988). 
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principles  of  Interaction  -  General  design  principles  that  relate  to  the 
characteristics  that  foster  (or  hinder)  operators’  acceptance  and  utilization  of 
adaptive  aiding  systems  (Rouse,  1988). 

random  omission  errors  -  An  information  processing  strategy  employed  by 
operators  under  high  information  input  parameters.  The  operator  omits  certain 
pieces  of  Information  on  a  random  basis  since  he  cannot  consider  all  decision- 
affecting  information  at  that  high  rate  of  input  (Sen,  1984). 

short-term  memory  support  -  The  use  of  display  technology  to  enhance  the 
human's  short-  term  memory  store.  For  example,  display  of  large  lists  of 
Information  should  be  maintained  when  the  operator  must  refer  to  it  often. 

situation  assessment  -  The  process  of  interpreting  relevant  data  from  the 
environment  in  an  effort  to  construct  an  accurate  representation  of  the  context 
based  on  current  information  needs  (Andes  and  Small,  1992;  conversation). 

stimulus-response  compatibility  (displays  and  controls)  -  The  relationship  of  the 
stimulus  code  to  the  response  code  (Barnes,  1981).  Examples  of  S-R 
compatibility  are  normal  population  stereotypes  (e.g,  light  switch  up  =  on  In  the 
United  States,  off  in  England). 

tailored  logic  -  The  employment  of  dynamic  logic  to  adapt  the  system  menus, 
command  sets,  and  possible  responses  based  on  various  environment  triggers 
(e.g.,  mission  mode,  situation  assessment).  (Barnes,  1985). 

task  requirements  centered  -  An  archetypal  aiding  system  design  philosophy 
concerned  primarily  with  the  functional  needs  for  which  the  system  was 
designed. 

task  execution  knowledge  -  Populated  knowledge  structures  concerned  with 
"how"  a  task  is  executed  from  within  the  system.  This  knowledge  is  specific  to 
the  execution  of  a  task  from  within  this  particular  system  (not  general):  there 
can  be  several  parallel  Instantiations  of  the  task  execution  knowledge 
depending  on  the  context  in  which  the  knowledge  Is  activated  (Andes,  1987). 

task  execution  goal  states  -  Knowledge  representation  schemes  containing 
desired  (or  predicted)  system  states  resulting  from  execution  of  a  system  task. 
Useful  in  determining  the  need  for  aiding,  or  for  continued  aid  assistance 
(Andes,  1987). 

timeliness  of  delivery  -  Refers  to  the  optimal  time  window  during  which  utility  of 
the  information  to  be  displayed  to  the  operator  is  at  a  peak.  This  is  based  on  the 
operator's  ability  to  interpret  the  information.  Diminishing  utility  is  associated 
with  the  age  of  the  information  (Andes,  1990). 

% 

user-initiated  adaptation  -  An  approach  to  adaptive  aid  invocation  where  the 
user  explicitly  requests  assistance  from  the  aid  (Rouse,  1988). 
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vigilance  support  -  Refers  to  a  desired  characteristic  of  the  aid  in  which  it 
monitors  input  streams  for  conditions  satisfying  predefined  situations  to  which 
the  user  should  be  alerted  in  order  to  reduce  the  perceptual  and  cognitive  load 
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